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Foreword 


The  Navy-wide  Personnel  Survey  (NPS)  is  a  major  product  of  the  Navy-Wide 
Survey  Program  (NWSP,  formerly  known  as  the  Navy  Personnel  Survey  System 
(NPSS)).  Administered  regularly  by  Navy  Personnel  Research,  Studies,  and 
Technology  (NPRST)  of  the  Bureau  of  Naval  Personnel  (BUPERS),  the  NPS 
focuses  on  quality  of  work-life  topics.  This  information  provides  valuable  metrics 
to  senior  leadership  and  program  managers  that  assist  in  the  evaluation  of  Navy 
quality  of  work  life,  current  personnel  policies,  and  programs.  NPS  is  also  one  of 
the  cornerstones  (along  with  the  Quality  of  Life  survey)  of  the  Navy’s  survey 
strategy.  The  survey  strategy  consists  of  two  omnibus,  multi-topic  Navy-wide 
surveys  to  provide  broad  work  and  personal  life  satisfaction  with  trends  over 
time.  This  information  is  augmented  by  quick  topic  specific  surveys  (Navy  Quick 
Polls),  focus  groups,  location  specific  assessments  (Rapid  Polls),  and  other 
methodologically  specific  studies.  One  of  the  important  functions  of  NPS  is  that 
its  items  and  Navy- wide  normative  data  serve  as  the  standard  validity  indicators 
for  virtually  all  other  Navy  personnel  surveys. 

The  2005  NPS  was  conducted  under  the  sponsorship  of  the  Chief  of  Naval 
Personnel  (CNP/Ni)  within  the  NWSP  funding  line.  Data  collection  began  in 
March  2005  and  concluded  in  June  2005.  A  briefing  was  prepared  in  July  2005 
and  the  results  were  presented  to  the  Chief  of  Naval  Personnel  and  the  Master 
Chief  Petty  Officer  of  the  Navy  in  September  2005. 

Because  of  the  pivotal  role  NPS  has  in  other  surveys,  the  data  were  analyzed  in 
detail  using  advanced  statistical  techniques  to  evaluate  the  psychometric 
properties  of  the  survey.  Existing  items  and  scales  were  evaluated,  and,  when 
warranted,  revised  scales  were  developed  and  further  evaluated.  In  addition, 
initial  work  was  conducted  to  develop  a  Navy  Climate  Index  and  a  Reenlistment 
Intention  Index.  The  results  of  these  analyses  are  presented  in  this  report. 

Any  questions  regarding  this  report  should  be  directed  to  the  NPS  Project 
Director,  Dr.  Kimberly  Whittam,  kimberly.whittam@navy.mil,  (901)  874-2321  or 
DSN  882-2321. 


DAVID  L.  ALDERTON,  Ph.D. 

Director 
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Executive  Summary 


One  of  the  most  comprehensive  sources  of  attitude  and  opinion  data  among 
Navy  personnel  is  the  Navy-wide  Personnel  Survey  (NPS).  In  the  current  study, 
the  psychometric  properties  of  existing  items  and  scales  on  the  NPS  were 
evaluated,  using  methods  such  as  Item  Response  Theory  (Lord,  1980)  and 
confirmatory  factor  analysis.  The  majority  of  items  and  scales  held  up  well. 
Revisions  were  recommended  to  6  of  the  18  scales  evaluated. 

Additionally,  initial  analysis  was  begun  to  develop  an  overall  measure  of  Navy 
climate.  The  Navy  Climate  Index  (NCI)  included  seven  scales  from  the  NPS:  (1) 
workplace  climate,  (2)  organizational  commitment,  (3)  morale,  (4)  job  security, 
(5)  communication,  (6)  fairness,  and  (7)  Navy  image.  The  index  demonstrated 
validity  based  on  a  second-order  confirmatory  factor  analysis  and  comparisons  of 
mean  scores  by  respondents’  ratings  of  job  satisfaction,  climate  in  the  Navy  and 
climate  in  their  current  command.  The  index  was  also  significantly  related  to 
intentions  to  re-enlist  and  to  have  a  full  career  (20  years  or  more)  in  the  Navy. 

Finally,  an  index  for  predicting  intentions  to  re-enlist  or  continue  at  the  next 
decision  point  was  also  developed.  This  index  was  developed  using  backwards 
stepwise  logistic  regression  to  identify  items  and  demographic  and  job 
characteristics  which  predicted  re-enlistment  and  continuation  intentions. 
Separate  models  were  developed  for  Sailors  in  their  first  term  of  service  and  those 
in  later  terms  of  service.  The  index  had  high  sensitivity  and  specificity  for 
predicting  retention  intentions  among  Sailors  in  their  first  term  of  service. 
However,  these  values  were  somewhat  lower  for  Sailors  in  later  terms  of  service. 

Future  areas  of  research  are  also  discussed.  These  recommendations  include 
exploring  the  suitability  of  utilizing  individual  items  as  proxies  for  longer  scales. 
In  some  circumstances,  shorter  scales  may  provide  benefits,  such  as  lower 
respondent  burden  and/or  reduced  time  and  costs  for  survey  administration, 
which  could  outweigh  a  minimal  loss  in  reliability  or  validity.  Additionally,  the 
data  from  NPS  2005  has  become  part  of  an  ongoing  NPRST  research  protocol 
that  follows  Sailors  through  time  to  see  whether  future  behavior  is  predicted  by 
attitudes  and  stated  career  intentions. 
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Background 


The  U.S.  Navy  faces  a  number  of  challenges  pertaining  to  the  management  of  the 
personnel  needs  of  its  officers  and  enlisted  personnel.  To  better  address  these 
challenges,  Navy  leadership  needs  accurate  metrics  that  capture  the  quality  of  Navy 
work  life  and  how  it  impacts  Sailors,  their  families,  and  the  Navy  organization  as  a 
whole.  In  an  era  characterized  by  changing  work  demands,  unpredictable  missions,  and 
possible  reductions  in  total  end-strength,  Navy  leadership  is  interested  in  measures  of 
the  overall  perception  of  the  Navy  experience.  This  perception  includes  workplace 
factors  such  as  job  satisfaction,  job  security,  compensation,  quality  of  life,  leadership, 
organizational  commitment,  fairness,  and  communication.  The  development  of  a  valid 
and  reliable  measure  of  Navy  Climate  and  the  evaluation  of  its  relationship  to  Sailor 
outcomes  (i.e.,  retention  intent,  retention  behavior)  would  provide  leadership  with 
insights  regarding  perceptions  of  Navy  life  as  a  whole. 

A  comprehensive  source  of  data  for  developing  the  Index  is  the  Navy-wide  Personnel 
Survey  (NPS),  a  long-standing  survey  conducted  by  researchers  at  Navy  Personnel 
Research,  Studies,  and  Technology  (NPRST).  The  NPS  was  first  administered  in  1990  at 
the  request  of  the  Chief  of  Naval  Personnel.  The  NPS  was  administered  annually  from 
1990  to  1998  and  at  longer  intervals  thereafter  with  surveys  being  fielded  in  2000, 

2003,  and  most  recently  in  2005.  Current  plans  are  to  re-administer  the  NPS  annually 
on  the  Internet. 

The  objective  of  the  NPS  is  to  measure  Sailor  satisfaction  with  Quality  of  Work  Life 
(QWL)  indicators  such  as  job  satisfaction,  organizational  commitment,  leadership 
satisfaction,  and  workplace  climate  and  their  effects  on  outcome  measures  such  as 
retention  intention.  The  NPS  also  captures  other  demographic  and  personnel  data  that 
may  not  be  accurately  captured  in  other  Navy  databases  (e.g.,  TEMPO,  career  plans, 
financial  status,  marital  status,  numbers  of  dependents)  and  satisfaction  with  other 
manpower  and  personnel  processes  (e.g.,  job  assignments  personnel  distribution 
(detailing),  performance  evaluations  and  promotions/advancement). 

NPRST  researchers  have  used  NPS  data  to  examine  a  wide  range  of  workplace 
factors  with  recent  efforts  focusing  on  modeling  workplace  factors  that  comprise  QWL 
(Janega  &  Whittam,  2004)  the  effect  of  operational  TEMPO  on  retention  plans 
(Olmsted  &  Whittam,  2004),  and  linking  retention  intentions  on  the  NPS  to  actual 
subsequent  retention  behaviors.  The  NPS  is  a  rich  source  of  Sailor  attitude  and 
perceptual  data  to  test  theoretical  models,  explore  the  measurement  of  new  concepts 
and  constructs,  and  for  the  present  study’s  goals. 

In  the  current  study,  the  psychometric  properties  of  existing  items  and  scales  on  the 
2005  NPS  were  evaluated  using  techniques,  such  as  item  response  theory  and 
confirmatory  factor  analysis.  Based  on  these  analyses,  revisions  to  the  existing  items  and 
scales  were  recommended.  Additionally,  a  Navy  Climate  index  was  developed  and  the 
relationship  between  index  scores  and  survey  respondents’  ratings  of  overall  climate,  job 
satisfaction,  and  morale  was  explored.  Finally,  a  short  index  using  items  from  the  NPS 
to  predict  re-enlistment/continuation  intentions  was  developed. 
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Existing  Items  and  Scales 


This  chapter  provides  an  overview  of  the  analyses  of  18  existing  scales  consisting  of 
101  items  from  the  2005  NPS.  The  2005  NPS  was  administered  on  the  Internet  between 
March  20  and  June  20,  2005  to  a  stratified  sample  of  16,372  active  duty  Sailors.  There 
were  a  total  of  3,610  usable  surveys  which,  after  adjusting  for  incomplete  surveys  and 
non-contacts,  resulted  in  an  overall  response  rate  of  26  percent  (unweighted)  and  37 
percent  (weighted  to  the  population).  To  evaluate  the  psychometric  properties  of  the 
scales,  descriptive  statistics  for  each  item  were  computed,  including  the  percentage  of 
respondents  who  indicated  they  agreed  or  disagreed  with  each  statement.  Items  with 
very  low  or  very  high  levels  of  agreement  may  not  be  informative  and  could  potentially 
be  removed  from  future  administrations  of  the  survey. 

Next,  the  dimensionality  of  the  scales  was  explored.  One-factor  confirmatory  factor 
analyses  were  conducted  to  test  whether  the  items  grouped  into  factors  corresponding 
to  the  scales.  The  model  fit  was  assessed  using  several  fit  indices,  including  the 
Comparative  Fit  Index  (CFI),  Tucker-Lewis  Fit  Index  (TLI),  and  the  Standardized  Root 
Mean  Square  Residual  (SRMR).  Ideally,  a  model  should  have  a  CFI  and  TLI  >  0.90  and 
an  SRMR  of  0.08  or  lower  for  an  acceptable  fit.  For  scales  with  three  items  or  fewer, 
exploratory,  rather  than  confirmatory,  factor  analyses  were  conducted  due  to  lack  of 
model  identification. 

Item  Response  Theory  (IRT)  analyses  were  then  conducted  to  further  explore  the 
properties  of  the  items.  Because  all  of  the  items  contained  five  ordered  response  options, 
item  parameter  estimates  were  computed  using  Samejima’s  graded  response  model 
(Samejima,  1969).  In  the  graded  model,  two  types  of  parameters  are  estimated  for  each 
item.  The  first  parameter  is  the  slope  or  a  parameter  which  quantifies  how  related  the 
item  is  to  the  construct  being  measured  by  the  scale.  In  addition  to  the  slope,  a  set  of 
threshold  or  b  parameters  are  estimated.  The  thresholds  locate  each  response  option 
along  the  continuum  of  the  underlying  construct.  In  other  words,  the  thresholds  for 
items  on  the  Organizational  Commitment  scale  would  indicate  the  approximate  level  of 
organizational  commitment  individuals  would  need  to  have  before  they  would  endorse 
the  corresponding  response  option.  The  number  of  thresholds  estimated  is  equal  to  the 
number  of  response  options  minus  one.  Therefore,  four  threshold  parameters  are 
estimated  for  items  on  the  NPS  that  contain  five  response  options  (e.g.,  strongly  agree  to 
strongly  disagree). 

In  addition  to  computing  the  IRT  item  parameters,  item  characteristic  curves  (ICC) 
were  plotted  for  each  item.  The  ICCs  present  the  probability  a  response  option  will  be 
endorsed  according  to  level  of  the  underlying  construct.  The  steepness  of  the  curves  is 
determined  by  the  item  slope  and  the  spread  of  the  curves  for  the  response  options  are 
determined  by  the  thresholds.  Ideally,  the  curve  for  each  response  option  should  have  a 
high  peak  and  none  of  the  curves  should  be  completely  overlapped  by  the  others. 
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For  each  item,  responses  of  “don’t  know”  or  “not  applicable”  were  recoded  as 
missing.  Scale  scores  were  computed  as  the  average  of  responses  to  the  items  on  the 
scale.  Before  computing  the  scale  scores,  items  were  reverse  coded  so  that  higher  scale 
scores  would  indicate  more  of  the  construct  being  measured  (e.g.,  higher  morale,  more 
job  security). 

The  confirmatory  factor  analyses  were  computed  using  the  MPlus  software  program 
(Muthen  &  Muthen,  2003)  while  the  IRT  analyses  were  computed  using  the  Multilog  for 
Windows  software  program  (Scientific  Software  International,  2003).  The  remaining 
analyses  were  conducted  using  SAS  version  9.  Because  methods  are  not  available  to 
incorporate  analysis  weights  into  the  IRT  analyses,  all  analyses  were  conducted  on  the 
unweighted  2005  NPS  data  to  ensure  the  results  are  comparable.  Below  is  a  brief 
summary  of  the  psychometric  properties  of  each  of  the  18  scales. 

Availability  of  Resources 

The  Availability  of  Resources  scale  includes  four  items  on  the  adequacy  of  personnel, 
tools,  spare  parts,  and  Navy  support  services.  Table  1  presents  the  percentage  of  survey 
respondents  who  indicated  that  they  agreed  or  strongly  agreed  with  each  of  the 
Availability  of  Resources  items.  Most  of  the  items  had  high  levels  of  agreement; 
however,  agreement  with  item  Q8D  was  noticeably  lower  than  the  others  with 
agreement  of  54  percent.  The  confirmatory  factor  analysis  suggested  that  all  four  items 
formed  a  single  underlying  factor. 

Table  1  displays  the  item  parameters  from  the  graded  response  IRT  model.  Ideally, 
items  should  have  slopes  of  1  or  higher  to  demonstrate  good  discrimination.  As  shown  in 
the  table,  all  four  items  on  the  scales  had  high  slopes,  indicating  that  they  can  effectively 
discriminate  between  respondents  with  high  vs.  low  perceptions  of  availability  of 
resources. 

The  threshold  parameters  provide  information  about  the  performance  of  the 
response  options  for  each  item.  For  example,  the  last  thresholds  for  items  Q8C  and  Q8D 
are  higher  than  those  for  items  Q8A  and  Q8B,  suggesting  that  a  higher  level  of 
satisfaction  with  availability  of  resources  is  required  before  endorsing  “strongly  agree” 
for  these  items. 

The  item  characteristic  curves  for  the  Availability  of  Resources  items  are  shown  in 
Figures  1-4.  Reviewing  the  curves  suggests  that  all  four  items  have  good  discrimination 
and  the  curves  for  the  response  options  are  generally  spread  across  the  horizontal  access 
representing  the  underlying  construct  (theta).  In  particular,  the  curves  for  item  Q8B  are 
especially  steep,  suggesting  very  good  discrimination. 
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Table  1 

Availability  of  Resources  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q8A 

Adequate  qualified  personnel 

76.1 

0.70 

1.96 

-2.45 

-1.45 

-0.94 

0.84 

Q8B 

Adequate  tools 

70.7 

0.79 

4.58 

-1.89 

-1.05 

-0.59 

0.90 

Q8C 

Adequate  spare  parts  and/or 
supplies 

54.4 

0.64 

2.28 

-1.87 

-0.90 

-0. 18 

1.43 

Q8D 

Adequate  Navy  support  services 

70.5 

0.50 

1.25 

-2.91 

-1.76 

-0.91 

1.43 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.98,  TLI  =  0.90,  SRMR  =  0.02. 
Correlated  errors  are  permitted  between  items  Q8B  and  Q8C. 
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Figure  2.  Item  characteristic  curves  for  items  on  Availability  of  Resources 

Scale  (Q8B:  Adequate  tools). 
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Item  characteristic  curves  for  items  on  Availability  of  Resources 
Scale  (Q8C:  Adequate  spare  parts  and/  or  supplies). 
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Figure  4.  Item  characteristic  curves  for  items  on  Availability  of  Resources 
Scale  (Q8D:  Adequate  Navy  support  services). 
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Morale 


The  item-level  statistics  for  the  Morale  scale  are  shown  in  Table  2.  The  percentage  of 
respondents  indicating  that  an  item  had  a  positive  or  strong  positive  effect  on  morale 
ranged  from  a  low  of  27  percent  for  item  Q10O  (TEMPO)  to  a  high  of  72  percent  for  item 
Q10F  (co-workers  and  shipmates).  The  factor  loadings  for  all  items  were  high  with 
values  of  0.50  and  higher  and  all  items  had  IRT  slopes  of  1  or  more.  The  ICCs  shown  in 
Figures  5-18  suggest  fairly  good  discrimination  for  the  items.  For  some  items  (e.g., 
Q10A,  Q10F,  Q10G),  the  curve  for  the  middle  response,  no  effect,  was  entirely  engulfed 
by  the  other  response  options,  suggesting  that  it  may  not  be  informative.  However, 
given  that  this  response  option  was  informative  for  some  items  (e.g.,  Q10C,  Q10D,  and 
Q10E);  it  is  suggested  that  they  be  retained  for  consistency  across  the  scale. 
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Table  2 
Morale  Scale 


Item  # 

Description 

% 

Positive/ 

Strong 

Positive 

Effect 

Factor 

Loading 

a 

1 RT  Parameters 

bl  b2  b3 

b4 

Q10A 

Advancement/  promotion 
opportunities 

55.8 

0.50 

1.20 

-2.90 

-1.22 

-0.25 

2.16 

Q10B 

Performance  evaluation  system 

51.9 

0.58 

1.50 

-2.59 

-1.11 

-0.06 

2.24 

Q10C 

Supply  of  spare  parts/ supplies 

30.7 

0.55 

1.27 

-2.69 

-0.93 

0.82 

3.14 

Q10D 

Quality  of  Navy  training  programs 

54.0 

0.62 

1.64 

-2.65 

-1.41 

-0.12 

2.21 

Q10E 

Quality  of  education  programs 

65.5 

0.57 

1.48 

-2.97 

-1.84 

-0.54 

1.72 

Q10F 

Co-  workers/  sh  i  pmates 

71.5 

0.58 

1.47 

-2.88 

-1.51 

-0.79 

1.35 

Q10G 

1  mmediate  supervisor 

67.9 

0.56 

1.57 

-2.36 

-1.34 

-0.62 

1.26 

Q10H 

Command  leadership 

60.7 

0.66 

1.89 

-1.83 

-0.91 

-0.32 

1.21 

Q10I 

Pace  of  work 

47.1 

0.68 

2.34 

-1.68 

-0.65 

0.08 

1.87 

Q10J 

Workload 

41.0 

0.69 

2.27 

-1.50 

-0.50 

0.26 

1.92 

Q10K 

Unit/workgroup  manning 

39.3 

0.66 

1.87 

-1.66 

-0.53 

0.33 

2.21 

Q10L 

Pay/ bonuses/ other  compensation 

49.7 

0.52 

1.16 

-2.74 

-1.59 

0.04 

2.29 

Q10M 

Amount  of  time  off 

55.8 

0.67 

1.77 

-1.69 

-0.81 

-0.17 

1.50 

Q10N 

Navy  support  services 

54.0 

0.50 

1.17 

-2.95 

-1.72 

-0.16 

2.36 

QlOO 

TEMPO 

27.0 

0.57 

1.30 

-1.87 

-0.72 

0.97 

2.57 

Q10P 

Performance  of  crew,  work  team  on 
exercises 

62.4 

0.57 

1.43 

-3.22 

-2.04 

-0.42 

1.88 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.92,  TLI  =  0.90,  SRMR  =  0.05. 

Correlated  errors  permitted  between  the  following  sets  of  items:  Q10  and  Q10B;  Q10D  and  Q10E;  Q10F,  Q10G,  and  Q10H;  and  Q10I  and  Q10J . 
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Figure  5. 1  tem  characteristic  curves  for  items  on  Morale  Scale 
(Q10A:  Advancement/  Promotion  opportunities). 
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Figure  6. 1  tem  characteristic  curves  for  items  on  Morale  Scale 
(Q10B:  Performance  evaluation  system). 
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Figure  7. 1  tem  characteristic  curves  for  items  on  Morale  Scale 
(Q10D:  Quality  of  Navy  training  programs). 
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Figure  8. 1  tem  characteristic  curves  for  items  on  Morale  Scale 
(Q10E:  Quality  of  education  programs). 
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Figure  9. 1  tem  characteristic  curves  for  items  on  Morale  Scale 
(Q10G:  I mmediate  supervisor). 
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Figure  10. 1  tem  characteristic  curves  for  items  on  Morale  Scale 
(Q10H:  Command  leadership). 
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Figure  11.  Item  characteristic  curves  for  items  on  Morale  Scale 

(Q10I :  Pace  of). 
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Figure  12. 1  tern  characteristic  curves  for  items  on  Morale  Scale 

(Q10J :  Workload). 
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Figure  13.  Item  characteristic  curves  for  items  on  Morale  Scale 
(Q10K:  Unity  Workgroup  manning). 
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Figure  14. 1  tern  characteristic  curves  for  items  on  Morale  Scale 
(Q10L:  Pay/  Bonuses/  Other  compensation). 


13 


1 


0.9 


-3  -2.25  -1.5  -0.75  0  0.75  1.5  2.25  3 


- Strong  negative  effect 

. Negative  effect 

- No  effect 

Positive  effect 
- Strong  positive  effect 


Theta 


Figure  15.  Item  characteristic  curves  for  items  on  Morale  Scale 
(Q10M:  Amount  of  time  off). 
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Figure  16. 1  tern  characteristic  curves  for  items  on  Morale  Scale 
(Q10N:  Navy  support  services). 
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Figure  17.  Item  characteristic  curves  for  items  on  Morale  Scale 

(QlOO:  TEMPO). 


-3  -2.25  -1.5  -0.75  0  0.75  1.5  2.25  3 


- Strong  negative  effect 

. Negative  effect 

- No  effect 

Positive  effect 
- Strong  positive  effect 


Theta 


Figure  18. 1  tern  characteristic  curves  for  items  on  Morale  Scale 
(Q10P:  Performance  of  crew/  work  team  on  exercises). 
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Gender  I  ntegration 


The  3-item  gender  integration  scale  is  shown  in  Table  3.  Eighty-one  percent  of 
respondents  indicated  that  the  leadership  in  the  organization  is  supportive  of  gender 
integration,  75  percent  indicated  that  women  have  the  ability  to  carry  out  combatant 
roles.  However,  a  smaller  percentage  (68  percent)  agreed  or  strongly  agreed  that  women 
are  being  successfully  integrated. 

All  three  items  had  high  IRT  slopes  and  factor  loadings.  In  particular,  item  Q12B 
(women  have  ability  to  carry  out  combatant  roles)  has  a  very  high  slope  (a=3-38), 
suggesting  that  it  may  be  central  to  perceptions  of  the  construct  of  gender  integration. 
The  high  discrimination  for  this  item  is  also  reflected  in  the  very  steep  ICCs  shown  in 
Figures  19-21. 
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Table  3 

Gender  integration  scale 


% 

1 RT  Parameters 

Item  # 

Description 

Satisfied/ 

Very 

Satisfied 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q12A 

Supportive  leadership 

Women  have  ability  to  carry  out 

81.0 

0.71 

1.78 

-2.91 

-2.20 

1.19 

0.46 

Q12B 

combatant  roles 

Women  are  being  successfully 

74.9 

0.86 

3.38 

-2.03 

-1.51 

0.76 

0.52 

Q12C 

integrated 

67.6 

0.82 

2.82 

-2.13 

-1.54 

0.58 

0.84 

Notes: 

Analyses  are  unweighted. 

Due  to  small  number  of  items,  exploratory  rather  than  confirmatory  factor  analysis  was  conducted.  Factor  eigenvalue  =  1.92. 
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Figure  19. 1  tem  characteristic  curves  for  items  on  Gender  I  ntegration  Scale 

(Q12A:  Supportive  leadership). 
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Figure  20. 1  tem  characteristic  curves  for  items  on  Gender  I  ntegration  Scale 
(Q12B:  Women  have  ability  to  carry  out  combatant  roles). 
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Figure  21.  Item  Characteristic  Curves  for  I  terns  on  Gender  I  ntegration  Scale 
(Q12C:  Women  are  being  successfully  integrated). 

Workplace  Climate 

The  nine  items  comprising  the  Workplace  Climate  scale  are  presented  in  Table  4. 
Levels  of  being  satisfied  or  very  satisfied  ranges  from  45  percent  for  item  Q13H 
(availability  of  spare  parts  and  supplies)  to  78  percent  for  item  Q13B  (amount  of 
responsibility  I  have  in  my  job).  While  most  items  had  high  slopes  and  factor  loadings, 
these  values  are  much  lower  for  item  Q13H  (availability  of  spare  parts  and  supplies). 
Visually,  this  difference  may  be  identified  by  examining  the  ICCs  in  Figures  22-29.  The 
curves  for  item  Q13H  are  much  flatter  than  those  for  the  other  items.  These  findings 
suggest  that  this  item  may  not  be  related  to  the  other  items  and  should  perhaps  be 
removed  in  future  NPS  administrations. 
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Table  4 

Workplace  Climate  Scale 


Item  # 

Description 

%  Agree/ 
Strongly 
Agree 

Factor 

Loading 

a 

1 RT  Parameters 

bl  b2  b3 

b4 

Q13A 

Amount  of  freedom  1  am  given  to 
do  my  job 

75.1 

0.67 

2.23 

-2.33 

-1.28 

-0.77 

0.69 

Q13B 

Amount  of  responsibility  1  have  in 
my  job 

77.8 

0.72 

3.26 

-2.17 

-1.30 

-0.77 

0.57 

Q13C 

Amount  of  challenge  in  my  job 

72.4 

0.79 

3.30 

-1.91 

-1.14 

-0.59 

0.66 

Q13D 

Opportunity  for  personal  growth 
and  development  on  the  job 

64.3 

0.81 

2.97 

-1.76 

-0.91 

-0.38 

0.87 

Q13E 

Feeling  of  accomplishment  1  get 
from  doing  my  job 

67.9 

0.79 

2.86 

-1.76 

-1.01 

-0.47 

0.73 

Q13F 

J  ob  security 

76.8 

0.51 

1.38 

-3.04 

-2.13 

-1.08 

0.67 

Q13G 

Physical  working  conditions  of  my 
work  site 

73.4 

0.50 

1.42 

-3.04 

-1.76 

-0.92 

1.14 

Q13H 

Availability  of  parts  and  supplies 

45.0 

0.35 

0.78 

-3.66 

-1.40 

0.30 

3.19 

Q13I 

Flexibility  in  dealing  with 
family/ personal  issues 

71.1 

0.52 

1.30 

-2.64 

-1.77 

-0.81 

0.81 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.93,  TLI  =  0.89,  SRMR  =  0.06 
Correlated  errors  permitted  between  items  Q13A,  Q13B,  and  Q13C. 
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Figure  22. 1  tern  characteristic  curves  for  items  on  Workplace  Climate  Scale 
(Q13A:  Amount  of  freedom  I  am  given  to  do  my  job). 
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Figure  23. 1  tern  characteristic  curves  for  items  on  Workplace  Climate  Scale 
(Q13B:  Amount  of  responsibility  I  have  in  my  job). 


21 


1 


0.9 

0.8 

0.7 

0.6 

>. 

5 

ro  0.5 

-Q 

O 

cl 

0.4 

0.3 

0.2 

0.1 

0 


'•/"N 

\  A  \ 

\  /  •.  \ 

■.  /  )  \  / 
\/  f\  \  / 

A  \  \  / 
/  x  \  V 


-3  -2.25  -1.5  -0.75  0  0.75  1.5  2.25  3 


- Very  dissatisfied 

. Dissatisfied 

- Neither  satisfied  nor  dissatisfied 

Satisfied 

- Very  satisfied 


Theta 


Figure  24.  Item  characteristic  curves  for  items  on  Workplace  Climate  Scale 
(Q13C:  Amount  of  challenge  in  my  job). 
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Figure  25. 1  tern  characteristic  curves  for  items  on  Workplace  Climate  Scale 
(Q13D:  Opportunity  for  personal  growth  and  development  on  the  job). 
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Figure  26.  Item  characteristic  curves  for  items  on  Workplace  Climate  Scale 
(Q13E:  Feeling  of  accomplishment  I  get  from  doing  my  job). 
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Figure  27. 1  tern  characteristic  curves  for  items  on  Workplace  Climate  Scale 

(Q13F:  Job  security). 
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Figure  28.  Item  characteristic  curves  for  items  on  Workplace  Climate  Scale 
(Q13H:  Availability  of  parts  and  supplies). 
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Figure  29. 1  tern  characteristic  curves  for  items  on  Workplace  Climate  Scale 
(Q13I :  Flexibility  in  dealing  with  family/  personal  issues). 
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Tempo 


Sixty-nine  percent  of  respondents  agreed  that  they  are  satisfied  with  the  amount  of 
time  spent  at  their  permanent  duty  station  and  61  percent  were  satisfied  with  their  time 
spent  on  shore  duty  (see  Table  5).  A  slightly  smaller  percentage  (57  percent)  was 
satisfied  with  their  time  spent  on  sea  duty.  All  three  items  had  good  psychometric 
properties  based  on  review  of  their  factor  loadings,  IRT  parameters,  and  ICCs  (see  Table 
5  and  Figures  30-32).  Items  Q19A  and  Q19B  had  slightly  better  discrimination  than 
item  Q19C. 
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Table  5 
Tempo  Scale 


%  I RT  Parameters 

Agree/ 


Item  # 

Description 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q19A 

Time  spent  at  permanent  duty 
station 

68.7 

0.86 

3.12 

-1.93 

-1.21 

-0.57 

0.66 

Q19B 

Time  spent  on  shore  duty 

61.0 

0.86 

3.13 

-1.79 

-1.19 

-0.36 

0.80 

Q19C 

Time  spent  on  sea  duty 

57.4 

0.74 

1.52 

-2.63 

-1.62 

-0.30 

1.32 

Notes: 

Analyses  are  unweighted. 

Due  to  small  number  of  items,  exploratory  rather  than  confirmatory  factor  analysis  was  conducted.  Factor  eigenvalue  =  2.02. 


to 

O' 


1 


0.9 


0.8 

0.7 


0.6 

> 


1q 

ro  0.5 

.q 

o 

CL 


0.4 


0.3 


0.2 

0.1 

0 


-3  -2.25  -1.5  -0.75  0  0.75  1.5  2.25 


- Strongly  disagree 

. Disagree 

- Neither 

Agree 

- Strongly  agree 


Theta 


Figure  30.  Item  characteristic  curves  for  items  on  Tempo  Scale 
(Q19A:  Time  spent  at  permanent  duty  station). 
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Figure  31.  Item  characteristic  curves  for  items  on  Tempo  Scale 
(Q19B:  Time  spent  on  shore  duty). 
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Figure  32.  Item  characteristic  curves  for  items  on  Tempo  Scale 
(Q19C:  Time  spent  on  sea  duty). 

I  mpact  on  Personal  Life 

The  brief  3-item  Impact  on  Personal  Life  scale  contains  items  with  a  wide  range  of 
agreement  levels  (see  Table  6).  While  only  29  percent  indicated  that  they  had  difficulty 
juggling  career  and  personal  life  (Q21C),  61  percent  indicated  that  their  career  causes 
significant  separation  from  family.  All  three  items  had  very  similar  factor  loadings 
(ranging  from  0.82  to  0.89),  high  IRT  slopes,  and  a  spread  of  threshold  parameters  (see 
Table  6  and  Figures  33-35). 
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Table  6 

I  mpact  on  Personal  Life  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q21A 

Career  gets  in  way  of  personal  life 
Career  causes  significant 

40.1 

0.89 

4.21 

-1.55 

-0.41 

0.26 

1.17 

Q21B 

separation  from  family 

Difficulty  juggling  career  and 

60.6 

0.82 

2.00 

-2.31 

-1.04 

-0.31 

0.92 

Q21C 

personal  life 

28.7 

0.85 

2.37 

-1.48 

-0.21 

0.65 

1.68 

Notes: 

Analyses  are  unweighted. 

Due  to  small  number  of  items,  exploratory  rather  than  confirmatory  factor  analysis  was  conducted.  Factor  eigenvalue  =  2.18. 
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Figure  33. 1  tern  characteristic  curves  for  items  on  I  mpact  on  Personal  Life 
Scale  (Q21A:  Career  gets  in  way  of  personal  life). 
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Figure  34. 1  tem  characteristic  curves  for  items  on  I  mpact  on  Personal  Life 
Scale  (Q21B:  Career  causes  significant  separation  from  family). 
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Figure  35. 1  tem  characteristic  curves  for  items  on  I  mpact  on  Personal  Life 
Scale  (Q21C:  Difficulty  juggling  career  and  personal  life). 

I  mmediate  Supervisor 

The  Immediate  Supervisor  scale  contains  the  six  items  shown  in  Table  7.  All  items  on 
this  scale  had  very  high  slopes  and  factor  loadings  as  reflected  in  their  steep  item 
characteristic  curves  (see  Figures  36-41).  In  particular,  the  slope  for  the  last  item  on  the 
scale,  Q23F  (Satisfied  with  immediate  supervisor),  was  extremely  high  (slope  =  9.14).  As 
expected,  this  item  may  be  a  particularly  salient  indicator  of  a  respondent’s  perceptions 
of  his/her  immediate  supervisor. 
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Table  7 

I  mmediate  Supervisor  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q23A 

Adequate  training/ expertise 

81.6 

0.76 

3.07 

-1.87 

-1.13 

-0.74 

0.54 

Q23B 

Deals  well  with  subordinates 

71.5 

0.91 

5.60 

-1.36 

-0.76 

-0.36 

0.65 

Q23C 

Deals  well  with  superiors 

Provides  adequate  support  and 

75.5 

0.79 

3.36 

-1.79 

-1.11 

-0.52 

0.71 

Q23D 

guidance 

Responsive  to  Sailor  needs  and 

68.1 

0.91 

5.59 

-1.34 

-0.74 

-0.27 

0.73 

Q23E 

concerns 

72.2 

0.88 

4.89 

-1.41 

-0.86 

-0.38 

0.69 

Satisfied  with  immediate 

Q23F 

supervisor 

72.6 

0.96 

9.14 

-1.24 

-0.75 

-0.38 

0.59 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.99,  TLI  =  0.98,  SRMR  =  0.02. 
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Figure  36. 1  tem  characteristic  curves  for  items  on  I  mmediate  Supervisor 
Scale  (Q23A:  Adequate  training/ expertise). 
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Figure  37.  Item  characteristic  curves  for  items  on  I  mmediate  Supervisor 
Scale  (Q23B:  Deals  well  with  subordinates). 
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Figure  38.  Item  characteristic  curves  for  items  on  I  mmediate  Supervisor 
Scale  (Q23C:  Deals  well  with  superiors). 
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Figure  39. 1  tern  characteristic  curves  for  items  on  I  mmediate  Supervisor 
Scale  (Q23D:  Provides  adequate  support  and  guidance). 
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Figure  40. 1  tem  characteristic  curves  for  items  on  I  mmediate  Supervisor 
Scale  (Q23E:  Responsive  to  Sailor  needs  and  concerns). 
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Figure  41.  Item  characteristic  curves  for  items  on  I  mmediate  Supervisor 
Scale  (Q23F:  Satisfied  with  immediate  supervisor). 
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Overall  Command  Leadership 

The  items  on  the  Overall  Command  Leadership  exhibited  good  psychometric 
properties  as  evidenced  by  their  high  factor  loadings,  IRT  slopes,  and  steep  ICCs  (see 
Table  8  and  Figures  42-47).  Similar  to  the  Immediate  Supervisor  scale,  the  rating  of 
overall  satisfaction  item,  Q24F  (Satisfied  with  command  leadership),  has  the  highest 
IRT  slope  (a=7-93),  suggesting  that  this  item  is  the  most  discriminating  measure  of 
perceptions  of  overall  command  leadership. 
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Table  8 

Overall  Command  Leadership  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q24A 

Adequate  training/ expertise 

81.2 

0.77 

2.95 

-0.71 

0.75 

1.35 

2.05 

Q24B 

Deals  well  with  subordinates 

68.4 

0.90 

5.39 

-0.89 

0.32 

0.80 

1.47 

Q24C 

Deals  well  with  superiors 

Provides  adequate  support  and 

74.8 

0.79 

3.38 

-0.90 

0.49 

1.26 

1.94 

Q24D 

guidance 

Responsive  to  Sailor  needs  and 

66.7 

0.90 

5.63 

-0.94 

0.27 

0.80 

1.36 

Q24E 

concerns 

68.4 

0.89 

5.13 

-0.88 

0.32 

0.83 

1.33 

Satisfied  with  command 

Q24F 

leadership 

67.9 

0.94 

7.93 

-0.84 

0.30 

0.78 

1.28 

CO 

-<1 


Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.98,  TLI  =  0.97,  SRMR  =  .02. 
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Figure  42. 1  tem  characteristic  curves  for  items  on  Overall  Command 
Leadership  Scale  (Q24A:  Adequate  training  and  expertise). 
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Figure  43. 1  tem  characteristic  curves  for  items  on  Overall  Command 
Leadership  Scale  (Q24B:  Deals  well  with  subordinates). 
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Figure  44. 1  tem  characteristic  curves  for  items  on  Overall  Command 
Leadership  Scale  (Q24C:  Deals  well  with  superiors). 
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Figure  45. 1  tem  characteristic  curves  for  items  on  Overall  Command 
Leadership  Scale  (Q24D:  Provides  adequate  support  and  guidance). 
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Figure  46. 1  tem  characteristic  curves  for  items  on  Overall  Command 
Leadership  Scale  (Q24E:  Responsive  to  Sailor  needs  and  concerns). 
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Figure  47.  Item  characteristic  curves  for  items  on  Overall  Command 
Leadership  Scale  (Q24F:  Satisfied  with  command  leadership). 
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Communication 

NPS  respondents  are  asked  to  indicate  their  level  of  agreement  with  the  six  items 
shown  in  Table  9.  Respondents  were  least  likely  to  agree  that  they  have  heard  rumors 
about  new  policies  (item  Q25F)  and  most  likely  to  agree  that  command  leadership 
communicates  positive  attitude  about  the  Navy  (item  Q25C).  While  the  first  five  items 
(Q25A-E)  performed  well,  the  last  item,  Q25F  (heard  rumors  about  new  policies)  had  a 
very  low  factor  loading  (loading=o.o8)  and  IRT  slope  (a=o.28).  As  shown  in  Figures 
48-53,  the  curves  for  this  item  are  nearly  flat.  The  thresholds  for  this  item  were  also  at 
the  extremes  (bi=-5-78  and  b4=io.22),  suggesting  that  a  respondent  must  be  very 
satisfied  or  dissatisfied  with  communication  to  endorse  any  response  option  other  than 
the  middle  one,  neither  agree  nor  disagree. 
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Table  9 

Communication  Scale 


Item  # 

Description 

% 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

1 RT  Parameters 

bl  b2  b3 

b4 

Q25A 

Navy  clearly  communicates  goals 
and  strategies 

59.1 

0.49 

1.81 

-2.54 

-1.09 

-0.32 

1.80 

Q25B 

Senior  leadership  keeps  Sailors 
informed 

60.2 

0.65 

2.66 

-2.09 

-0.94 

-0.32 

1.56 

Q25C 

Command  leadership 
communicates  positive  attitude 
about  Navy 

72.3 

0.66 

1.86 

-2.57 

-1.65 

-0.78 

1.08 

Q25D 

Command  leadership  keeps  me 
informed  of  Navy  policies 

66.8 

0.86 

3.09 

-2.15 

-1.17 

-0.49 

1.15 

Q25E 

Someone  in  chain  of  command 

talked  about  new  career  initiatives 

49.9 

0.65 

1.72 

-1.68 

-0.58 

-0.03 

1.54 

Q25F 

Heard  rumors  about  new  policies 

48.3 

0.08 

0.28 

-5.78 

-0.30 

3.91 

10.22 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.99,  TLI  =  0.97,  SRMR  =  0.03. 
Correlated  errors  permitted  between  items  Q25  A  and  Q25B. 
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Figure  48. 1  tern  characteristic  curves  for  items  on  Communication  Scale 
(Q25A:  Navy  clearly  communicates  goals  and  strategies). 
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Figure  49. 1  tern  characteristic  curves  for  items  on  Communication  Scale 
(Q25B:  Senior  leadership  keeps  Sailors  informed). 
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Figure  50. 1  tem  characteristic  curves  for  items  on  Communication  Scale 
(Q25C:  Command  leadership  communicates  positive  attitude  about  navy). 


1  T 


-3  -2.25  -1.5  -0.75  0  0.75  1.5  2.25  3 


- Strongly  disagree 

. Disagree 

- Neither 

Agree 

- Strongly  agree 


Theta 


Figure  51.  Item  characteristic  curves  for  items  on  Communication  Scale 
(Q25D:  Command  leadership  keeps  me  informed  of  Navy  policies). 
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Figure  52. 1  tern  characteristic  curves  for  items  on  Communication  Scale 
(Q25E:  Someone  in  chain  of  command  talked  about  new  career  initiatives). 
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Figure  53. 1  tern  characteristic  curves  for  items  on  Communication  Scale 
(Q25F:  Heard  rumors  about  new  policies). 
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J  ob  Security 

To  assess  job  security,  respondents  were  asked  the  six  items  shown  in  Table  to. 
While  the  first  three  items  (Q26A-C)  had  high  slopes  and  factor  loadings,  the  remaining 
three  items  (Q26D-F)  had  low  factor  loadings  (ranging  from  0.14  to  0.33)  and  slopes 
(ranging  from  0.28  to  0.78).  As  shown  in  Figures  54-59,  the  ICCs  for  items  Q26D  to 
Q26F  are  almost  flat.  These  results  suggest  that  perhaps  this  scale  should  be  divided 
into  two  scales  with  the  first  including  items  Q26A-C  and  the  second  including  items 
Q26D-F. 


46 


Table  10 

J  ob  Security  Scale 


%  I RT  Parameters 

Agree/ 


Item  # 

Description 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q26A 

Feel  positive  about  future  Navy 

career 

56.6 

0.71 

2.15 

-1.78 

-0.86 

-0.22 

1.12 

Q26B 

Navy  is  doing  all  it  can  to  protect 
my  job  security 

44.6 

0.81 

3.13 

-1.64 

-0.75 

0. 12 

1.38 

Q26C 

Future  in  Navy  appears  secure 

65.6 

0.86 

3.69 

-1.67 

-0.98 

-0.41 

0.89 

Q26D 

Willing  to  change 

rating/ designator  to  stay  in  Navy 

36.6 

0.14 

0.28 

-4.62 

-1.05 

2.01 

6.83 

Q26E 

Concerned  fellow  Sailors  may  lose 
their  jobs 

46.3 

0.21 

0.49 

-3.86 

-0.33 

2.41 

6.22 

Q26F 

Concerned  future  policy  changes 
will  hurt  job 

45.0 

0.33 

0.78 

-2.46 

-0.29 

1.74 

4.15 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.96,  TLI  =  0.93,  SRMR  =  0.05. 
Correlated  errors  permitted  between  items  Q26E  and  Q26F. 
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Figure  54. 1  tem  characteristic  curves  for  items  on  J  ob  Security  Scale  (Q26A:  I 
feel  positive  about  my  future  Navy  career) . 
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Figure  55.  Item  characteristic  curves  for  items  on  J  ob  Security  Scale  (Q26B: 
The  Navy  is  doing  all  it  can  to  protect  my  job  security). 
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Figure  56. 1  tem  characteristic  curves  for  items  on  J  ob  Security  Scale  (Q26C: 
My  future  in  the  Navy  appears  secure  as  long  as  I  do  a  good  job). 
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Figure  57.  Item  characteristic  curves  for  items  on  J  ob  Security  Scale  (Q26D:  I 
would  be  willing  to  change  my  rating/  designator  if  it  was  the  only  way  I 

could  stay  in  the  Navy). 
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Figure  58.  Item  characteristic  curves  for  items  on  J  ob  Security  Scale  (Q26E:  I 
am  concerned  that  some  of  my  fellow  Sailors  may  soon  lose  their  jobs). 
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Figure  59. 1  tern  characteristic  curves  for  items  on  J  ob  Security  Scale  (Q26F:  I 
am  concerned  that  future  policy  changes  will  hurt  my  job). 
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Fairness 


The  four  items  on  the  Fairness  scale  performed  well  (see  Table  11).  Reviewing  the 
ICCs  shows  that  all  four  items  had  steep  curves  (see  Figures  60-63).  In  addition,  the 
curves  for  the  response  options  are  spread  across  the  continuum  of  theta  with  no  curve 
being  engulfed  by  another,  indicating  that  each  response  option  is  informative  and 
should  be  retained. 
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Table  11 
Fairness  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q27A 

Navy  personnel  policies  seem  fair 
Navy  policies  retain  best  quality 

57.0 

0.72 

2.27 

-1.99 

-1.01 

-0.21 

1.62 

Q27B 

Sailors 

T rust  Navy  to  look  out  for  my  best 

40.9 

0.78 

2.70 

-1.37 

-0.44 

0.24 

1.68 

Q27C 

interests 

Confident  policies  affecting  size  of 
Navy  will  be  administered  fairly 

30.4 

0.78 

2.73 

-1.20 

-0.28 

0.55 

1.95 

Q27D 

and  consistently 

39.8 

0.78 

2.85 

-1.50 

-0.65 

0.27 

1.87 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.99,  TLI  =  0.98,  SRMR  =  0.01. 
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Figure  60. 1  tem  characteristic  curves  for  items  on  Fairness  Scale 
(Q27A:  Navy  personnel  policies  seem  fair). 
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Figure  61. 1  tem  characteristic  curves  for  items  on  Fairness  Scale 
(Q27B:  Navy  policies  retain  best  quality  Sailors). 
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Figure  62.  Item  characteristic  curves  for  items  on  Fairness  Scale 
(Q27C:  Trust  Navy  to  look  out  for  my  best  interests). 
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Figure  63. 1  tern  characteristic  curves  for  items  on  Fairness  Scale 
(Q27D:  Confident  policies  affecting  size  of  Navy  will  be  administered  fairly 

and  consistently). 
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Navy  I  mage 


Levels  of  agreement  with  the  items  comprising  the  Navy  Image  scale  range  from  a 
low  of  36  percent  for  item  Q28F  (Comments  I  hear  about  the  Navy  from  fellow  Sailors 
are  usually  positive)  to  a  high  of  70  percent  for  item  Q28B  (I  talk  about  the  Navy  to 
friends  as  a  good  organization).  Interestingly,  the  items  on  this  scale  also  appear  to  split 
into  two  with  the  first  three  items  (Q28A-C)  having  very  high  slopes  and  factor  loadings 
while  the  other  four  items  (Q28D-G)  have  lower  slopes  and  factor  loadings  (see  Table 
12).  The  differences  in  discrimination  between  these  two  sets  of  items  are  also  reflected 
in  differences  in  the  steepness  of  their  ICCs  (see  Figures  64-70).  However,  the  last  four 
items  do  reach  acceptable  levels  of  discrimination,  suggesting  that  they  may  be  retained 
on  the  scale. 
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Table  12 

Navy  I  mage  Scale 


Item  # 

Description 

% 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

1 RT  Parameters 

bl  b2  b3 

b4 

Q28A 

1  would  encourage  others  to  join 
the  Navy 

60.9 

0.90 

5.69 

-1.29 

-0.72 

-0.22 

0.84 

Q28B 

1  talk  about  Navy  to  friends  as  a 
good  organization 

69.7 

0.89 

5.37 

-1.57 

-0.99 

-0.42 

0.77 

Q28C 

1  would  recommend  the  Navy  as  a 
good  place  to  work 

63.1 

0.93 

7.39 

-1.40 

-0.83 

-0.26 

0.84 

Q28D 

1  would  wear  civilian  clothing  with 
Navy  logos 

56.9 

0.49 

1.39 

-1.83 

-0.89 

-0.24 

1.43 

Q28E 

1  nformation  1  hear  about  Navy 
from  non- Navy  sources  is  usually 
positive 

55.2 

0.50 

1.42 

-2.72 

-1.38 

-0.15 

2.20 

Q28F 

Comments  1  hear  about  Navy 
from  fellow  Sailors  are  usually 
positive 

35.5 

0.64 

1.88 

-1.63 

-0.40 

0.48 

2.46 

Q28G 

Navy  of  tomorrow  will  be  better 
than  today 

36.4 

0.52 

1.35 

-2.09 

-1.06 

0.56 

2.19 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.96,  TLI  =  0.94,  SRMR  =  0.05. 
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Figure  64. 1  tem  characteristic  curves  for  items  on  Navy  I  mage  Scale 
(Q28A:  I  would  encourage  others  to  join  the  Navy). 


- Strongly  disagree 

. Disagree 

- Neither 

Agree 

- Strongly  agree 


Figure  65.  Item  characteristic  curves  for  items  on  Navy  I  mage  Scale 
(Q28B:  I  talk  about  Navy  to  friends  as  a  good  organization). 
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Figure  66. 1  tem  characteristic  curves  for  items  on  Navy  I  mage  Scale 
(Q28C:  I  would  recommend  the  Navy  as  a  good  place  to  work). 
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Figure  67.  Item  characteristic  curves  for  items  on  Navy  I  mage  Scale 
(Q28D:  I  would  wear  civilian  clothing  with  Navy  logos). 
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Figure  68.  Item  characteristic  curves  for  items  on  Navy  I  mage  Scale  (Q28E 
I  nformation  I  hear  about  Navy  from  non-Navy  sources  is  usually  positive). 
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Figure  69. 1  tern  characteristic  curves  for  items  on  Navy  I  mage  Scale  (Q28F 
I  nformation  I  hear  about  Navy  from  fellow  Sailors  is  usually  positive). 
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Figure  70. 1  tem  characteristic  curves  for  items  on  Navy  I  mage  Scale 
(Q28G:  Navy  of  tomorrow  will  be  better  than  today). 

Organizational  Commitment 

The  Organizational  Commitment  scale  is  comprised  of  five  items  shown  in  Table  13. 
The  items  have  variation  in  levels  of  agreement.  Almost  three-quarters  of  respondents 
indicated  that  the  Navy  has  personal  meaning  for  them  (item  Q37A)  while  only  41 
percent  indicated  that  they  could  not  easily  become  attached  to  another  organization 
(Q37D).  Reviewing  the  ICCs  (see  Figures  71-75),  factor  loadings,  and  IRT  parameters 
suggests  that  all  five  items  are  effectively  able  to  discriminate  between  personnel  with 
low  vs.  high  levels  of  organizational  commitment. 
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Table  13 

Organizational  Commitment  Scale 


%  I RT  Parameters 

Agree/ 


Item  # 

Description 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q37A 

Navy  has  personal  meaning  for 

me 

73.1 

0.82 

3.41 

-1.96 

-1.27 

-0.59 

0.51 

Q37B 

1  feel  like  1  'm  "part  of  the  family" 
in  the  Navy 

60.4 

0.87 

4.11 

-1.57 

-0.86 

-0.24 

0.78 

Q37C 

1  feel  "emotionally  attached"  to 
the  Navy 

51.9 

0.89 

4.86 

-1.29 

-0.66 

-0.02 

0.87 

Q37D 

1  could  not  easily  become 
attached  to  another  organization 

40.7 

0.75 

2.52 

-1.38 

-0.55 

0.26 

1.18 

Q37E 

1  feel  a  strong  sense  of  belonging 
in  the  Navy 

58.3 

0.91 

5.33 

-1.40 

-0.81 

-0. 17 

0.81 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.99,  TLI 

=  0.98,  SRMR  = 

0.02. 
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Figure  71. 1  tern  characteristic  curves  for  items  on  Organizational 
Commitment  Scale  (Q37A:  Navy  has  personal  meaning  for  me). 
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Figure  72.  Item  characteristic  curves  for  items  or  Organizational 
Commitment  Scale  (Q38B:  I  feel  like  I 'm  part  of  the  family  in  the  Navy.) 
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Figure  73. 1  tem  characteristic  curves  for  items  on  Organizational 
Commitment  Scale  (Q37C:  I  feel  emotionally  attached  to  the  Navy). 
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Figure  74. 1  tem  characteristic  curves  for  items  on  Organizational 
Commitment  Scale  (Q37D:  I  could  not  easily  become  attached  to  another 

organization). 
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Figure  75. 1  tem  characteristic  curves  for  items  on  Organizational 
Commitment  Scale  (Q37E:  I  feel  a  strong  sense  of  belonging  in  the  Navy). 

Advancement/  Promotion 

The  item-level  statistics  for  the  Advancement/Promotion  scale  are  shown  in  Table 
14.  The  items  varied  greatly  in  their  levels  of  agreement.  While  82  percent  of  Sailors 
indicated  that  they  have  a  clear  understanding  of  the  advancement/promotion  system 
(Q38A),  only  34  percent  indicated  that  only  the  most  qualified  Sailors  get  promoted 
(Q38C).  As  shown  in  Figures  76-79,  the  ICCs  for  items  Q38A  (clear  understanding  of 
advancement/promotion  system)  and  Q38D  (expect  to  be  promoted  within  current 
term)  were  clustered  on  the  lower  end  of  theta.  In  addition,  the  curves  for  item  Q38D 
(“Expect  to  be  promoted  within  current  term”)  are  flatter,  suggesting  that  this  item  may 
not  be  related  to  the  other  items  on  the  scale. 
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Table  14 

Advancement/  Promotion  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Clear  understanding  of 

Q38A 

advancement/ promotion  system 
Satisfied  with 

81.7 

0.42 

0.96 

-4.26 

-2.60 

-1.73 

0.93 

Q38B 

advancement/ promotion  system 
The  most  qualified  Sailors  get 

43.4 

0.89 

4.34 

-1.14 

-0.32 

0.17 

1.29 

Q38C 

promoted 

Expect  to  be  promoted  within 

34.0 

0.71 

2.23 

-1.13 

-0.09 

0.50 

1.75 

Q38D 

current  term 

59.9 

0.26 

0.55 

-4.28 

-2.60 

-0.75 

2.02 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.99,  TLI  =  0.98,  SRMR  =  0.02. 
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Figure  76. 1  tem  characteristic  curves  for  items  on  Advancement/  Promotion 
Scale  (Q38A:  Clear  understanding  of  advancement/  promotion  system). 
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Figure  77.  Item  characteristic  curves  for  items  on  Advancement/  Promotion 
System  Scale  (Q38B:  Satisfied  with  advancement/  promotion  system). 
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Figure  78. 1  tern  characteristic  curves  for  items  on  Advancement/  Promotion 
Scale  (Q38C:  The  most  qualified  Sailors  get  promoted). 
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Figure  79. 1  tern  characteristic  curves  for  items  on  Advancement/  Promotion 
Scale  (Q38D:  Expect  to  be  promoted  within  current  term). 
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Performance  Evaluations/  Fitness  Reports 

NPS  respondents  were  asked  seven  questions  concerning  performance  evaluations 
and  fitness  reports  (EVAL/FITREPs).  As  shown  in  Table  15  and  Figures  80-87,  items 
Q39E  (last  promotion  recommendation  was  fair)  and  Q29B  (last  EVAL/FITREP  was 
fair/accurate)  had  the  highest  slopes  and  factor  loadings,  suggesting  that  these  items 
may  be  salient  indicators  of  respondents’  perceptions  concerning  performance 
evaluations  and  fitness  reports 
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Table  15 

Performance  Evaluations/  Fitness  Reports  Scale 


Item  # 

Description 

% 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

1 RT  Parameters 

bl  b2  b3 

b4 

Q39A 

Clear  understanding  of  present 
system 

84.0 

0.48 

1.45 

-3.25 

-2.05 

-1.40 

0.66 

Q39B 

Last  EVAL/FITREP  was 
fair/accurate 

71.6 

0.86 

4.20 

-1.53 

-1.00 

-0.58 

0.63 

Q39C 

Last  EVAL/FITREP  was  conducted 
in  a  timely  manner 

75.9 

0.69 

2.73 

-1.90 

-1.27 

-0.79 

0.69 

Q39D 

Able  to  submit  input  at  my  last 
EVAL/FITREP 

78.2 

0.62 

2.18 

-2.12 

-1.45 

-0.91 

0.55 

Q39E 

Last  promotion  recommendation 
was  fair 

72.0 

0.84 

3.86 

-1.64 

-1.15 

-0.59 

0.54 

Q39F 

Satisfied  with  present 

EVAL/FITREP  system 

52.8 

0.59 

1.79 

-1.75 

-0.90 

-0.15 

1.45 

Q39G 

Most  qualified  and  deserving 

Sailors  score  highest  on  the 
EVALs/FITREPs 

37.0 

0.48 

1.21 

-1.58 

-0.39 

0.54 

2.29 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.96,  TLI  =  0.93,  SRMR  =  0.04. 
Correlated  errors  permitted  between  items  Q39F  and  Q39G. 
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Figure  80.  Item  characteristic  curves  for  items  on  Performance  Evaluations/ 
Fitness  Reports  Scale  (Q39A:  Clear  understanding  of  present  system). 
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Figure  81. 1  tern  characteristic  curves  for  items  on  Performance  Evaluations/ 
Fitness  Reports  Scale  (Q39B:  Last  EVAIV  FITREP  was  fair/  accurate). 
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Figure  82.  Item  characteristic  curves  for  items  on  Performance  Evaluations/ 
Fitness  Reports  Scale  (Q39C:  Last  EVAIV  FITREP  was  conducted  in  a  timely 

manner). 
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Figure  83. 1  tern  characteristic  curves  for  items  on  Performance  Evaluations/ 
Fitness  Reports  Scale  (Q39D:  Able  to  submit  input  at  my  last  EVAL/  FITREP). 
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Figure  84.  Item  characteristic  curves  for  items  on  Performance  Evaluations/ 
Fitness  Reports  Scale  (Q39E:  Last  promotion  recommendation  was  fair). 
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Figure  85. 1  tern  characteristic  curves  for  items  on  performance  Evaluations/ 
Fitness  Reports  Scale  (Q39F:  Satisfied  with  present  EVAL/  FITREP  system). 
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Figure  86. 1  tem  characteristic  curves  for  items  on  performance  Evaluations/ 
Fitness  Reports  Scale  (Q39G:  Most  qualified  and  deserving  Sailors  score 

highest  on  the  EVALs/  FI  TREPs) . 


Recognition 

The  NPS  includes  two  items  concerning  adequate  recognition  for  accomplishments 
(see  Table  16).  Respondents  indicated  slightly  more  agreement  that  they  were 
adequately  recognized  for  accomplishments  on  EVAL/FITREPs  (63%  agree/strongly 
agree)  than  that  they  were  recognized  with  awards  (50%  agree/strongly  agree).  Both 
items  demonstrated  good  psychometric  properties  and  had  steep  ICCs  (see  Figures  87- 
88). 
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Table  16 

Recognition  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2  b3 

b4 

Adequately  recognized  for 
accomplishments  on 

Q40A 

EVALs/FITREPs 

Adequately  recognized  for 

62.5 

0.91 

3.31 

-1.66 

-0.89  -0.35 

0.89 

Q40B 

accomplishments  with  awards 

49.6 

0.91 

3.08 

-1.36 

-0.64  -0.04 

1.16 

Notes: 

Analyses  are  unweighted. 

Due  to  small  number  of  items,  exploratory  rather  than  confirmatory  factor  analysis  was  conducted.  Factor  eigenvalue  =  1.65. 
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Figure  87.  Item  characteristic  curves  for  items  on  Recognition  Scale 
(Q40A:  Adequately  recognized  for  accomplishments  on  EVALs/  FITREPs). 
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Figure  88. 1  tern  characteristic  curves  for  items  on  Recognition  Scale 
(Q40B:  Adequately  recognized  for  accomplishments  with  awards). 
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Career  Development 

The  four-item  Career  Development  scale  is  shown  in  Table  17.  While  all  four  items 
had  slopes  with  a  value  of  at  least  one,  the  two  items  concerning  counseling  and 
guidance  (items  Q41C  and  Q41D)  had  noticeably  higher  slopes  with  values  of  4.79  and 
3.96,  respectively  (see  Figures  89-92).  In  contrast,  the  slopes  for  items  Q41A  and  Q41C 
were  1.35  and  1.16,  respectively.  This  differential  in  slopes  suggests  that  perhaps  these 
items  may  be  split  into  two  scales. 
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Table  17 

Career  Development  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Clearly  defined  path  for  my 

04 1A 

designator,  rating,  or  community 
Sufficient  progress  in  my 

63.7 

0.52 

1.35 

-2.55 

-1.45 

-0.54 

1.38 

Q41B 

advancement 

Given  adequate 
counseling/guidance  by  my 

69.9 

0.45 

1.16 

-3.30 

-2.01 

-0.87 

1.31 

•'4 

Q4 1C 

immediate  supervisor 

Given  adequate 

counseling/guidance  by  my  career 

48.4 

0.90 

4.79 

-1.34 

-0.64 

0.02 

1.23 

Q41D 

counselor 

44.7 

0.86 

3.96 

-1.27 

-0.61 

0.11 

1.32 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.88,  TLI  =  0.64,  and  SRMR  =  0.10. 


1  -T 


0.9  - 
0.8  - 
0.7  - 


0  -I - 1 - T '  ~r — f — “i  r - 1 - 1 - 1 - 1 - 1 — -r— — i — ~T  "  i  ~  ~  i  *  1  r  ~~1  ‘ 1  T  ■  r~- 

-3  -2.25  -1.5  -0.75  0  0.75  1.5  2.25  3 


- Strongly  disagree 

. Disagree 

- Neither 

Agree 

- Strongly  agree 


Theta 


Figure  89.  Item  characteristic  curves  for  items  on  Career  Development  Scale 
(Q41A:  Clearly  defined  path  for  my  designator,  rating,  or  community). 
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Figure  90. 1  tern  characteristic  curves  for  items  on  Career  Development  Scale 
(Q41B:  Sufficient  progress  in  my  advancement). 
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Figure  91. 1  tern  characteristic  curves  for  items  on  Career  Development  Scale 
(Q41C:  Given  adequate  counseling/ guidance  by  my  immediate  supervisor). 
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Figure  92. 1  tern  characteristic  curves  for  items  on  Career  Development  Scale 
(Q41D:  Given  adequate  counseling/  guidance  by  my  career  counselor). 
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Detailing 


The  final  scale  is  the  7-item  Detailing  scale  shown  in  Table  18.  Percentages  of 
respondents  indicating  that  they  agreed  or  strongly  agreed  with  the  items  ranged  from 
36  percent  for  item  Q42D  (detailer  is  an  advocate  for  my  needs/desires)  to  61  percent 
for  item  Q42G  (satisfied  with  current  assignment).  Similar  to  the  Immediate  Supervisor 
and  Command  Leadership  scales,  the  overall  rating  item,  Q42F  (satisfied  with  my 
detailer)  had  the  highest  level  of  discrimination  (a=7-57).  In  contrast,  item  Q42G 
(satisfied  with  current  assignment)  had  a  much  lower  slope  (a=i.io)  and, 
correspondingly,  had  flatter  ICCs  (see  Figures  93-98). 
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Table  18 
Detailing  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q42A 

Satisfied  with  detailing  process 
Clear  understanding  of  detailing 

45.2 

0.77 

2.68 

-1.47 

-0.69 

0.14 

1.77 

Q42B 

process 

Detailer  responds  in  a  timely 

55.4 

0.53 

1.48 

-2.31 

-1.03 

-0.14 

1.75 

Q42C 

manner 

48.3 

0.82 

3.36 

-1.53 

-0.92 

0.05 

1.24 

Detailer  is  an  advocate  for  my 

Q42D 

needs/ desires 

Detailer  is  receptive  to  resolving 

35.5 

0.91 

5.18 

-1.23 

-0.65 

0.36 

1.44 

Q42E 

conflicts 

38.7 

0.92 

5.62 

-1.32 

-0.79 

0.28 

1.46 

Q42F 

Satisfied  with  my  detailer 

43.7 

0.94 

7.57 

-1.29 

-0.85 

0.14 

1.24 

Q42G 

Satisfied  with  current  assignment 

60.9 

0.44 

1.10 

-2.39 

-1.57 

-0.46 

1.44 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.97,  TLI  =  0.96,  and  SRMR  =  0.03. 


- Strongly  disagree 

. Disagree 

- Neither 

Agree 

- Strongly  agree 


Theta 


Figure  93. 1  tem  characteristic  curves  for  items  on  Detailing  Scale 
(Q42A:  Satisfied  with  detailing  process). 
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Figure  94. 1  tem  characteristic  curves  for  items  on  Detailing  Scale 
(Q42B:  Clear  understanding  of  detailing  process). 
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Figure  95. 1  tem  characteristic  curves  for  items  on  Detailing  Scale 
(Q42D:  Detailer  is  an  advocate  for  my  needs/  desires). 
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Figure  96. 1  tem  characteristic  curves  for  items  on  Detailing  Scale 
(Q42E:  Detailer  is  receptive  to  resolving  conflicts). 
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Figure  97. 1  tern  characteristic  curves  for  items  on  Detailing  Scale 
(Q42F:  Satisfied  with  my  detailer). 
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Figure  98. 1  tern  characteristic  curves  for  items  on  Detailing  Scale 
(Q42G:  Satisfied  with  current  assignment). 
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Revised  Scales 


Based  on  the  analyses  of  the  existing  scales,  the  following  six  scales  were  identified 
which  may  be  revised  to  improve  their  psychometric  properties:  (l)  workplace  climate, 
(2)  communication,  (3)  job  security,  (4)  advancement/promotion,  (5)  career 
development,  and  (6)  detailing.  Revisions  are  proposed  to  each  of  these  scales  and  the 
psychometric  properties  of  the  revised  scales  are  evaluated. 

The  reliability  and  validity  of  the  final  set  of  scales  were  then  evaluated,  including  the 
six  revised  scales  and  the  12  original  scales.  Reliability  is  the  consistency  of  a  scale 
across  repeated  measurements.  One  measure  of  reliability  is  the  internal  consistency  of 
the  scale  or  how  related  the  items  on  the  scale  are  to  each  other.  Cronbach’s  alphas  was 
used  to  assess  the  internal  consistency  reliability  of  the  NPS  scales  (Cronbach,  1951).  A 
common  rule  of  thumb  is  to  require  an  alpha  of  0.70  or  higher  for  group  level 
comparisons  and  an  alpha  of  0.90  or  higher  for  individual-level  decisions. 

Validity  is  generally  described  as  the  extent  to  which  a  scale  measures  what  it  is 
designed  to  measure.  To  assess  the  construct  validity  of  the  scales,  scale  scores  for 
groups  that  should  vary  on  the  construct  being  measured  were  compared.  Specifically, 
analyses  of  variance  was  conducted  to  compare  mean  scores  for  groups  classified 
according  to  their  responses  to  the  following  items: 

•  Qg  :  How  would  you  rate  the  overall  morale  of  your  present  command? 

•  Q11:  Considering  everything,  how  satisfied  are  you  with  your  Navy  job? 

•  Q29A:  How  would  you  rate  Navy  tone? 

•  Q65:  Considering  everything,  how  satisfied  are  you  with  Navy  life? 

It  is  hypothesized  that  Sailors  with  the  following  characteristics  would  have  higher 
scores  (i.e.,  more  positive  responses)  on  each  of  the  scales: 

•  Higher  morale, 

•  Greater  j ob  satisfaction, 

•  Higher  ratings  of  Navy  tone,  and 

•  Greater  satisfaction  with  Navy  life. 

Revised  Workplace  Climate  Scale 

Based  on  the  item  analyses  described  in  the  Workplace  Climate  Section,  it  is 
recommend  that  item  Q13H  (Availability  of  parts  and  supplies)  be  removed  from  the 
Workplace  Climate  scale.  This  item  had  a  low  slope  parameter.  It  also  seems  to  differ 
from  the  other  items  in  terms  of  content.  While  the  other  items  on  the  scale  address 
more  motivational  or  interpersonal  characteristics  of  climate  (e.g.,  Q13D:  Opportunity 
for  personal  growth  and  development),  this  item  concerns  more  tangible,  concrete  items 
(parts  and  supplies).  However,  because  the  item  appears  to  be  a  salient  concern  for 
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Sailors  as  demonstrated  by  the  45  percent  agreement  rate,  it  is  recommend  that  the  item 
be  retained  on  the  survey  and  analyzed  separately  from  the  other  items  addressing 
Workplace  Climate. 

To  evaluate  the  revised  Workplace  Climate  scale,  a  l-factor  confirmatory  factor 
model  was  analyzed.  As  shown  in  Table  19,  this  model  fit  well.  With  the  removal  of  item 
Q13H,  all  of  the  Workplace  Climate  items  now  have  acceptable  properties. 
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Table  19 

Revised  Workplace  Climate  Scale 


Item  # 

Description 

% 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

1 RT  Parameters 

bl  b2  b3 

b4 

Q13A 

Amount  of  freedom  1  am  given  to 
do  my  job 

75.1 

0.66 

2.19 

-2.36 

-1.29 

-0.77 

0.70 

Q13B 

Amount  of  responsibility  1  have  in 
my  job 

77.8 

0.72 

3.28 

-2.18 

-1.30 

-0.77 

0.57 

Q13C 

Amount  of  challenge  in  my  job 

72.4 

0.80 

3.37 

-1.91 

-1.13 

-0.59 

0.66 

Q13D 

Opportunity  for  personal  growth 
and  development  on  the  job 

64.3 

0.82 

2.97 

-1.76 

-0.91 

-0.38 

0.87 

Q13E 

Feeling  of  accomplishment  1  get 
from  doing  my  job 

67.9 

0.80 

2.87 

-1.76 

-1.01 

-0.47 

0.74 

Q13F 

J  ob  security 

76.8 

0.51 

1.36 

-3.07 

-2.15 

-1.09 

0.68 

Q13G 

Physical  working  conditions  of  my 
work  site 

73.4 

0.49 

1.37 

-3.12 

-1.81 

-0.94 

1.16 

Q13I 

Flexibility  in  dealing  with 
family/ personal  issues 

71.1 

0.51 

1.26 

-2.70 

-1.81 

-0.83 

0.82 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.95,  TLI  =  0.92,  SRMR  =  0.05. 
Correlated  errors  permitted  between  items  Q13A,  Q13B,  and  Q13C. 


Revised  Communication  Scale 

Similar  to  the  Workplace  Climate  scale,  the  Communication  scale  contained  one  item 
that  did  not  fit  well  with  the  other  items.  As  described  in  the  Communications  Section, 
item  Q25F  (heard  rumors  about  new  policies)  had  very  low  levels  of  discrimination 
ability  (a=o.28).  This  item  also  appears  to  be  different  from  the  others  on  the  scale  in 
terms  of  content.  While  items  Q25A-E  concern  communication  from  Navy  leadership, 
item  Q25F  may  involve  communication  from  informal  sources,  such  as  shipmates  or  co¬ 
workers. 

Item  Q25F  was  removed  and  the  analysis  was  conducted  for  the  revised 
Communication  scale.  As  shown  in  Table  20,  the  remaining  items  grouped  into  one 
factor.  All  of  the  items  on  the  revised  scale  had  high  factor  loadings  and  slopes,  and  a 
spread  of  threshold  parameters. 
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Table  20 

Revised  Communication  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Navy  clearly  communicates  goals 

Q25A 

and  strategies 

Senior  leadership  keeps  Sailors 

59.1 

0.49 

1.77 

-2.57 

-1.10 

0.33 

1.82 

Q25B 

informed 

Command  leadership 
communicates  positive  attitude 

60.2 

0.65 

2.58 

-2.11 

-0.95 

0.32 

1.58 

Q25C 

about  Navy 

Command  leadership  keeps  me 

72.3 

0.66 

1.87 

-2.57 

-1.64 

0.77 

1.08 

Q25D 

informed  of  Navy  policies 

Someone  in  chain  of  command 

66.8 

0.86 

3.16 

-2.14 

-1.16 

0.49 

1.14 

Q25E 

talked  about  new  career  initiatives 

49.9 

0.65 

1.74 

-1.67 

-0.58 

0.03 

1.53 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  1.00,  TLI  =  1.00,  SRMR  =  0.01. 
Correlated  errors  permitted  between  items  Q25A  and  Q25B. 


Revised  J  ob  Security  Scale 

Based  on  the  item  analyses  of  the  Job  Security  scale,  it  is  recommended  that  items 
Q26D-F  be  removed  from  the  Job  Security  scale.  These  items  demonstrated  poor 
discrimination  based  on  their  IRT  slopes.  As  shown  in  Table  21,  removal  of  these  items 
results  in  a  brief  scale  containing  only  items  with  high  levels  of  discrimination. 
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Table  21 

Revised  J  ob  Security  Scale 


%  I RT  Parameters 


Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q26A 

Feel  positive  about  future  Navy 

career 

56.6 

0.84 

2.24 

-1.74 

-0.84 

-0.21 

l.ll 

Q26B 

Navy  is  doing  all  it  can  to  protect 
my  job  security 

44.6 

0.87 

2.84 

-1.67 

-0.77 

0. 12 

1.42 

Q26C 

Future  in  Navy  appears  secure 

65.6 

0.89 

4.14 

-1.62 

-0.96 

-0.41 

0.87 

Notes: 

Analyses  are  unweighted. 

Due  to  small  number  of  items,  exploratory  rather  than  confirmatory  factor  analysis  was  conducted.  Factor  eigenvalue  =  2.25. 


Revised  Advancement/  Promotion  Scale 

The  original  Advancement/Promotion  scale  contains  four  items  shown  in  Table  22. 
Of  these  four  items,  one  item,  Q38D  (expect  to  be  promoted  within  current  term) 
appeared  to  be  unrelated  to  the  others  based  on  its  low  factor  loading  (loading=o.26) 
and  slope  (a=o.55).  While  items  Q38A-C  address  understanding  of  and  satisfaction  of 
the  advancement  and  promotion  system,  item  Q38D  relates  to  personal  achievement  or 
expectations.  Therefore,  it  is  recommended  that  this  item  be  removed  from  the  scale. 
The  item-level  statistics  for  a  revised  Advancement/Promotion  scale,  excluding  item 
Q38D,  are  shown  in  Table  22. 
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Table  22 

Revised  Advancement/  Promotion  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Clear  understanding  of 

Q38A 

advancement/ promotion  system 
Satisfied  with 

81.7 

0.64 

0.95 

-4.29 

-2.62 

-1.74 

0.94 

Q38B 

advancement/ promotion  system 
The  most  qualified  Sailors  get 

43.4 

0.88 

4.89 

-1.12 

-0.32 

0. 16 

1.27 

Q38C 

promoted 

34.0 

0.84 

2.13 

-1.15 

-0.09 

0.52 

1.78 

Notes: 

Analyses  are  unweighted. 

Due  to  small  number  of  items,  exploratory,  rather  than  confirmatory,  factor  analyses  were  conducted.  Factor  eigenvalue  =  1.88. 


Revised  Career  Development  Scales 

As  described  earlier,  the  pattern  of  slopes  and  factor  loadings  for  this  scale  seem  to 
suggest  that  the  items  may  be  measuring  two  different  constructs  and  could  be  divided 
into  separate  scales.  A  2-factor  confirmatory  factor  model  was  tested  for  which  items 
Q41A  and  Q41B  load  on  one  factor,  which  is  referred  to  as  Career  Progression  and  items 
Q41C  and  Q41D  load  on  another  factor,  referred  to  as  Counseling/Guidance.  As  shown 
in  Table  23,  the  two-factor  model  fit  well  and  all  of  the  items  have  high  slopes  and 
loadings  on  their  respective  scales. 
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Table  23 

Revised  Career  Development  Scales 


%  I RT  Parameters 


Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Factor  1 

Career  progression 

Q41A 

Clearly  defined  path  for  my 
designator,  rating,  or  community 

63.7 

0.80 

2.14 

-2.02 

-1.21 

-0.49 

1.09 

Q41B 

Sufficient  progress  in  my 
advancement 

69.9 

0.67 

2.94 

-2.09 

-1.34 

-0.62 

0.85 

Factor  2 

Counseling/  guidance 

Q41C 

Given  adequate  counseling/ 
guidance  by  my  immediate 
supervisor 

48.4 

0.91 

3.80 

-1.44 

-0.66 

0.04 

1.28 

Q41D 

Given  adequate  counseling/ 
guidance  by  my  career  counselor 

44.7 

0.86 

5.56 

-1.24 

-0.59 

0.14 

1.26 

Notes: 

Analyses  are  unweighted. 

Fit  indices  for  two-factor  confirmatory  factor  model:  CFI  =  1.00,  TLI  =  0.99,  and  SRMR  =  0.01. 


Revised  Detailing  Scale 

Our  final  set  of  revisions  concerns  the  Detailing  scale.  As  described  earlier,  it 
appeared  that  item  Q42G  (satisfied  with  current  assignment)  was  less  discriminating 
than  the  other  items  based  on  its  lower  IRT  slope  (slope=i.io)  and  flatter  ICCs. 
Furthermore,  in  contrast  to  the  other  items  on  the  scale,  this  item  is  not  specifically 
related  to  detailing.  Item  Q42G  was  removed  and  the  IRT  and  confirmatory  factor 
analyses  were  rerun.  As  shown  in  Table  24,  the  six  remaining  items  on  the  scale  have 
high  factor  loadings  and  IRT  slopes,  indicating  good  discrimination. 
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Table  24 

Revised  Detailing  Scale 


% 

1 RT  Parameters 

Item  # 

Description 

Agree/ 

Strongly 

Agree 

Factor 

Loading 

a 

bl 

b2 

b3 

b4 

Q42A 

Satisfied  with  detailing  process 
Clear  understanding  of  detailing 

45.2 

0.76 

2.67 

-1.48 

-0.70 

0.13 

1.75 

Q42B 

process 

Detailer  responds  in  a  timely 

55.4 

0.52 

1.47 

-2.33 

-1.04 

-0.15 

1.75 

Q42C 

manner 

48.3 

0.82 

3.33 

-1.54 

-0.93 

0.05 

1.23 

Detailer  is  an  advocate  for  my 

Q42D 

needs/ desires 

Detailer  is  receptive  to  resolving 

35.5 

0.91 

5.34 

-1.23 

-0.65 

0.35 

1.42 

Q42E 

conflicts 

38.7 

0.92 

5.71 

-1.33 

-0.79 

0.28 

1.44 

Q42F 

Satisfied  with  my  detailer 

43.7 

0.93 

7.39 

-1.30 

-0.86 

0.13 

1.24 

Notes: 

Analyses  are  unweighted. 

Confirmatory  factor  model  fit  indices:  CFI  =  0.98,  TLI  =  0.96,  and  SRMR  =  0.03. 


Final  NPS  Scales 


Table  25  displays  the  Cronbach’s  alphas  for  the  final  set  of  scales,  including  the  seven 
revised  scales  and  the  12  original  scales.  All  but  one  of  the  scales  had  alphas  of  0.70  or 
higher,  indicating  acceptable  internal  consistency  for  group-level  comparisons.  The 
remaining  scale,  the  Career  Development:  Career  Progression  scale,  had  an  alpha  close 
to  0.70  with  a  value  of  0.68. 

The  comparisons  of  the  mean  scale  scores  across  the  overall  ratings  items  are 
presented  in  Tables  26  to  29.  The  comparisons  for  all  scales  were  highly  significant  (p  < 
.0001).  The  patterns  of  means  fit  our  hypotheses.  As  expected,  higher  scale  scores  were 
found  for  those  with  greater  morale,  job  satisfaction,  ratings  of  Tone,  and  satisfaction 
with  Navy  life,  supporting  the  construct  validity  of  the  scales. 


Table  25 

Cronbach  alphas  for  final  scales 


Scale 

Items 

Alpha 

Availability  of  Resources 

Q8A-D 

0.77 

Morale 

Q10A-P 

0.90 

Gender  1  ntegration 

Q12A-C 

0.77 

Workplace  Climate 

Q13A-G,  1 

0.87 

Tempo 

Q19A-C 

0.76 

Impact  on  Personal  Life 

Q21A-C 

0.81 

Immediate  Supervisor 

Q23A-F 

0.95 

Overall  Command  Leadership 

Q24A-F 

0.95 

Communication 

Q25A-E 

0.81 

J  ob  Security 

Q26A-C 

0.83 

Fairness 

Q27A-D 

0.85 

Navy  1  mage 

Q28A-G 

0.87 

Organizational  Commitment 

Q37A-E 

0.93 

Advancement/Promotion 

Q38A-C 

0.70 

Performance  Evaluations/ 

Q39A-G 

0.84 

Fitness  Reports 

Recognition 

Q40A-B 

0.79 

Career  Development:  Career  progression 

Q41A-B 

0.68 

Career  Development:  Counseling/guidance 

Q41C-D 

0.88 

Career  Development 

Q41A-D 

0.79 

Detailing 

Q42A-F 

0.92 

Note:  Analyses  are  unweighted 
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Table  26 

Mean  final  scale  scores  by  respondent  ratings  of  overall  morale 


Overall  Morale 


Scale 

Very  Low 

Low 

Medium 

High 

Very  High 

F-  statistic 

p- value 

Availability  of  Resources 

2.72 

3.19 

3.56 

4.02 

4.44 

368.82 

<.0001 

Morale 

2.43 

2.83 

3.28 

3.69 

4.12 

705.51 

<.0001 

Gender  1  ntegration 

3.29 

3.65 

3.89 

4. 18 

4.50 

138.16 

<.0001 

Workplace  Climate 

2.90 

3.33 

3.77 

4.20 

4.57 

444.04 

<.0001 

Tempo 

3.03 

3.28 

3.60 

3.93 

4.23 

135.95 

<.0001 

1  mpact  on  Personal  Life 

3.83 

3.54 

3.23 

2.92 

2.79 

86.20 

<.0001 

1  mmediate  Supervisor 

3.01 

3.41 

3.80 

4.23 

4.58 

198.02 

<.0001 

Overall  Command  Leadership 

2.57 

3.09 

3.70 

4.25 

4.67 

505.58 

<.0001 

Communication 

2.64 

3.05 

3.44 

3.82 

4. 19 

301.19 

<.0001 

J  ob  Security 

2.59 

3.03 

3.34 

3.73 

4.06 

149.28 

<.0001 

Fairness 

2.32 

2.66 

3.01 

3.37 

3.65 

154.60 

<.0001 

Navy  1  mage 

2.45 

2.89 

3.34 

3.75 

3.97 

305.07 

<.0001 

Organizational  Commitment 

2.52 

3.01 

3.46 

3.90 

4. 13 

189.17 

<.0001 

Advancement/ Promotion 

2.70 

2.87 

3.22 

3.52 

3.79 

106.34 

<.0001 

Performance 

3.06 

3.31 

3.64 

3.93 

4.20 

152.94 

<.0001 

Evaluations/ Fitness  Reports 
Recognition 

2.51 

2.89 

3.35 

3.77 

3.92 

138.27 

<.0001 

Career  Development: 

3.08 

3.34 

3.66 

3.90 

4. 12 

84.75 

<.0001 

Career  Progression 

Career  Development: 

2.27 

2.71 

3.11 

3.57 

3.88 

151.33 

<.0001 

Counseling/ Guidance 

Detailing 

2.68 

2.93 

3.17 

3.52 

3.70 

95.06 

<.0001 

Note:  Analyses  are  unweighted. 
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Table  27 

Mean  Final  Scale  Scores  by  Respondent  Ratings  of  Overall  J  ob  Satisfaction 


Overall  J  ob  Satisfaction 
Neither 
Satisfied 


Scale 

Very 

Dissatisfied 

Dissatisfied 

nor 

Dissatisfied 

Satisfied 

Very 

Satisfied 

F-statistic 

p- value 

Availability  of  Resources 

2.95 

3.20 

3.47 

3.76 

4.05 

163.27 

<.0001 

Morale 

2.50 

2.84 

3.14 

3.46 

3.79 

385.17 

<.0001 

Gender  1  ntegration 

3.19 

3.62 

3.77 

4.03 

4.32 

130.36 

<.0001 

Workplace  Climate 

2.59 

3. 14 

3.55 

3.99 

4.50 

791.29 

<.0001 

Tempo 

2.79 

3.24 

3.39 

3.76 

4.15 

187.83 

<.0001 

1  mpact  on  Personal  Life 

3.98 

3.68 

3.40 

3.09 

2.72 

134.78 

<.0001 

1  mmediate  Supervisor 

2.94 

3.38 

3.68 

4.01 

4.33 

154.92 

<.0001 

Overall  Command  Leadership 

2.78 

3.20 

3.53 

3.92 

4.29 

224.25 

<.0001 

Communication 

2.64 

3.00 

3.29 

3.61 

3.96 

242.52 

<.0001 

J  ob  Security 

2.09 

2.71 

3.14 

3.56 

4.09 

364.49 

<.0001 

Fairness 

2.04 

2.52 

2.89 

3.20 

3.54 

209.43 

<.0001 

Navy  1  mage 

2.14 

2.74 

3.08 

3.55 

3.98 

509.35 

<.0001 

Organizational  Commitment 

2.05 

2.86 

3.13 

3.68 

4.21 

372.76 

<.0001 

Advancement/  Promotion 

2.57 

2.86 

2.98 

3.34 

3.76 

140.85 

<.0001 

Performance 

Evaluations/ Fitness  Reports 

3.02 

3.23 

3.41 

3.77 

4.15 

206.07 

<.0001 

Recognition 

2.41 

2.78 

3.09 

3.52 

4.00 

182.80 

<.0001 

Career  Development: 

Career  Progression 

2.77 

3.26 

3.41 

3.77 

4.18 

169.68 

<.0001 

Career  Development: 
Counseling/ Guidance 

2.08 

2.61 

2.94 

3.32 

3.75 

165.71 

<.0001 

Detailing 

2.56 

2.87 

2.99 

3.34 

3.68 

122.91 

<.0001 

Note:  Analyses  are  unweighted. 
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Table  28 

Mean  Final  Scale  Scores  by  Respondent  Ratings  of  Navy  Tone 


Navy  Tone 


Scale 

Very  Low 

Low 

Medium 

High 

Very  High 

F-  statistic 

p- value 

Availability  of  Resources 

2.92 

3.24 

3.64 

3.92 

4.29 

145.73 

<.0001 

Morale 

2.59 

2.95 

3.31 

3.63 

4.05 

275.40 

<.0001 

Gender  1  ntegration 

3.34 

3.67 

3.91 

4. 16 

4.51 

87.77 

<.0001 

Workplace  Climate 

2.84 

3.38 

3.84 

4. 13 

4.55 

255.73 

<.0001 

Tempo 

2.91 

3.35 

3.63 

3.89 

4.31 

99.22 

<.0001 

1  mpact  on  Personal  Life 

4.01 

3.65 

3.20 

2.89 

2.67 

104.20 

<.0001 

1  mmediate  Supervisor 

3.04 

3.51 

3.89 

4. 12 

4.48 

93.57 

<.0001 

Overall  Command  Leadership 

2.98 

3.33 

3.75 

4.08 

4.55 

140.33 

<.0001 

Communication 

2.55 

2.99 

3.46 

3.84 

4.39 

297.92 

<.0001 

J  ob  Security 

2.19 

2.74 

3.35 

3.88 

4.53 

338.03 

<.0001 

Fairness 

1.95 

0.35 

2.98 

3.55 

4.27 

445.84 

<.0001 

Navy  1  mage 

2.05 

2.61 

3.35 

3.89 

4.38 

684.24 

<.0001 

Organizational  Commitment 

2.06 

2.82 

3.50 

3.95 

4.49 

306.81 

<.0001 

Advancement/ Promotion 

2.58 

2.67 

3.19 

3.65 

4.16 

203.22 

<.0001 

Performance 

Evaluations/ Fitness  Reports 

3.06 

3.28 

3.65 

3.94 

4.35 

140.51 

<.0001 

Recognition 

2.44 

2.82 

3.38 

3.74 

4.20 

133.52 

<.0001 

Career  Development: 

Career  Progression 

2.92 

3.25 

3.65 

3.94 

4.43 

123.24 

<.0001 

Career  Development: 
Counseling/ Guidance 

2.23 

2.64 

3.18 

3.51 

4.10 

130.26 

<.0001 

Detailing 

2.52 

2.78 

3.22 

3.53 

4.00 

137.06 

<.0001 

Note:  Analyses  are  unweighted. 
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Table  29 

Mean  Final  Scale  Scores  by  Respondent  Ratings  of  Overall  Satisfaction  with  Navy  Life 


Overall  Satisfaction  with  Navy  Life 
Neither 
Satisfied 


Scale 

Very 

Dissatisfied 

Dissatisfied 

nor 

Dissatisfied 

Satisfied 

Very 

Satisfied 

F- 

statistic 

p- value 

Availability  of  Resources 

2.88 

3.22 

3.49 

3.73 

4.07 

135.25 

<.0001 

Morale 

2.45 

2.87 

3.17 

3.43 

3.78 

285.91 

<.0001 

Gender  1  ntegration 

3.08 

3.57 

3.74 

4.04 

4.28 

115.73 

<.0001 

Workplace  Climate 

2.58 

3. 18 

3.58 

3.96 

4.45 

488.67 

<.0001 

Tempo 

2.53 

3. 10 

3.38 

3.75 

4.28 

259.49 

<.0001 

1  mpact  on  Personal  Life 

4.30 

3.91 

3.48 

3.08 

2.52 

252.27 

<.0001 

1  mmediate  Supervisor 

3.01 

3.41 

3.70 

3.96 

4.33 

109.36 

<.0001 

Overall  Command  Leadership 

2.89 

3.23 

3.56 

3.88 

4.30 

169.23 

<.0001 

Communication 

2.62 

2.96 

3.29 

3.58 

4.01 

223.13 

<.0001 

J  ob  Security 

2.01 

2.60 

3.03 

3.56 

4.18 

391.17 

<.0001 

Fairness 

1.93 

2.38 

2.81 

3.16 

3.74 

290.82 

<.0001 

Navy  1  mage 

1.82 

2.56 

2.98 

3.56 

4.12 

788.07 

<.0001 

Organizational  Commitment 

1.63 

2.56 

2.95 

3.73 

4.43 

711.63 

<.0001 

Advancement/ Promotion 

2.44 

2.69 

2.94 

3.33 

3.93 

212.74 

<.0001 

Performance 

Evaluations/ Fitness  Reports 

2.88 

3.17 

3.39 

3.76 

4.23 

230.86 

<.0001 

Recognition 

2.20 

2.71 

3.03 

3.51 

4.09 

208.82 

<.0001 

Career  Development: 

Career  Progression 

2.58 

3.12 

3.35 

3.79 

4.26 

222.23 

<.0001 

Career  Development: 
Counseling/ Guidance 

2.12 

2.57 

2.90 

3.31 

3.75 

138.85 

<.0001 

Detailing 

2.43 

2.69 

2.94 

3.32 

3.86 

194.30 

<.0001 

Note:  Analyses  are  unweighted. 


Navy  Climate  I  ndex  (NCI ) 


After  establishing  a  set  of  reliable  and  valid  scales,  a  subset  of  these  scales  was  used 
to  develop  an  overall  measure  of  Navy  Climate.  This  construct  has  previously  been 
described  as  “Navy  Tone”  but  Climate  describes  the  construct  in  a  more  familiar  fashion 
because  it  can  be  thought  of  as  a  single  metric  that  captures  the  Navy’s  overall 
“temperature”  or  “climate.”  As  shown  in  Figure  99,  Navy  Climate  was  conceptualized  as 
including  seven  underlying  constructs  measured  by  the  NPS:  (1)  workplace  climate,  (2) 
organizational  commitment,  (3)  morale,  (4)  job  security,  (5)  communication,  (6) 
fairness,  and  (7)  Navy  image.  The  factor  structure  of  the  Navy  Climate  Index  was  tested 
using  a  higher-order  confirmatory  factor  analysis  and  assessed  its  construct  validity  by 
conducing  analyses  of  variance  to  compare  NCI  scores  by  items  on  the  NPS  which  asks 
respondents’  to  rate  overall  Navy  Tone  (i.e.,  Climate). 
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Figure  99.  Constructs  comprising  Navy  Climate  I  ndex. 


Development  of  the  Navy  Climate  I  ndex 

To  test  the  conceptual  model  of  Navy  Climate  shown  in  Figure  99,  a  second-order 
confirmatory  factor  model  was  conducted  using  the  items  and  scales  corresponding  to 
each  of  the  constructs  underlying  Navy  Climate.  Figure  100  presents  a  path  diagram  for 
the  second-order  confirmatory  factor  model.  In  this  model,  the  following  were  tested: 

(a)  if  the  relevant  items  load  on  each  of  the  scales  (Workplace  Climate,  Organizational 
Commitment,  Morale,  Job  Security,  Communication,  Fairness,  and  Navy  Image)  and  (b) 
if  these  items  form  a  single  construct  representing  overall  Navy  Climate.  As  shown  in 
the  figure,  correlated  errors  are  permitted  between  two  pairs  of  scales:  (a)  Workplace 
Climate  and  Fairness,  and  (b)  Navy  Image  and  Organizational  Commitment. 
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The  fit  indices  suggest  that  the  model  has  an  acceptable  fit,  indicating  that  the  seven 
scales  may  be  combined  into  an  overall  Navy  Climate  Index  (CFI  =  0.91,  TLI  =  0.91, 
SRMR  =0.06).  Table  30  presents  the  factor  loadings.  With  the  exception  of 
Organizational  Commitment,  the  scales  have  similar  loadings  with  values  of 
approximately  0.80. 

The  factor  loadings  for  the  scales  were  used  to  compute  scores  for  the  NCI.  Weights 
were  computed  for  each  scale  based  on  the  scale’s  factor  loading  divided  by  the  sum  of 
the  factor  loadings.  Specifically,  Climate  index  scores  were  computed  using  the  following 
formula  where  scale  scores  are  the  mean  of  the  corresponding  items: 

Climate  =  25*((o.i5*Morale  +  0.15* Workplace  Climate  +  o. ^Communication 
+  o.i5*Job  Security  +  o.i5*Fairness  +  o.i4*Navy  Image 
+  o.i2*Organizational  Commitment)-!.) 

Index  scores  range  from  o  to  too  with  higher  values  indicating  a  more  positive  Navy 
climate. 

Table  31  presents  mean  NCI  by  demographic  characteristics.  Significantly  higher 
NCI  scores  were  found  for  those  who  are  male,  have  a  higher  education,  are  married, 
have  children  under  age  21  in  their  household,  are  an  officer  or  warrant  officer,  are  not 
in  their  first  term  of  service,  and  not  currently  deployed. 
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Figure  100.  Path  diagram  for  second-order  confirmatory  factor  model  of  Navy  Climate  I  ndex. 
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Table  30 

Results  of  second-order  confirmatory  factor  model 
for  Navy  Climate  I  ndex 


Path 

Standardized 

Coefficient 

Scales 

Morale 

Q10A 

0.53 

Q10B 

0.60 

Q10C 

0.53 

Q10D 

0.62 

Q10E 

0.58 

Q10F 

0.60 

Q10G 

0.59 

Q10H 

0.68 

Q10I 

0.62 

Q10J 

0.61 

Q10K 

0.59 

Q10L 

0.54 

Q10M 

0.63 

Q10N 

0.48 

QlOO 

0.50 

Q10P 

0.59 

Workplace  Climate 

Q13A 

0.65 

Q13B 

0.72 

Q13C 

0.76 

Q13D 

0.81 

Q13E 

0.80 

Q13F 

0.55 

Q13G 

0.50 

Q13I 

0.56 

Communication 

Q25A 

0.56 

Q25B 

0.69 

Q25C 

0.69 

Q25D 

0.78 

Q25E 

0.65 

J  ob  Security 

Q26A 

0.83 

Q26B 

0.70 

Q26C 

0.72 
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Table  30 

Results  of  second-order  confirmatory  factor  model 
for  Navy  Climate  I  ndex 


Path 

Standardized 

Coefficient 

Fairness 

Q27A 

0.75 

Q27B 

0.77 

Q27C 

0.81 

Q27D 

0.77 

Navy  1  mage 

Q28A 

0.90 

Q28B 

0.89 

Q28C 

0.92 

Q28D 

0.50 

Q28E 

0.50 

Q28F 

0.65 

Q28G 

0.51 

Organizational  Commitment 

Q37A 

0.83 

Q37B 

0.89 

Q37C 

0.88 

Q37D 

0.75 

Q37E 

0.92 

Navy  Climate  1  ndex 

Morale 

0.81 

Workplace  Climate 

0.82 

Communication 

0.80 

J  ob  Security 

0.85 

Fairness 

0.83 

Navy  1  mage 

0.79 

Organizational  Commitment 

0.66 

Model  Fit  1  ndices 

Comparative  Fit  1  ndex 

0.91 

Tucker-Lewis  1  ndex 

0.91 

Standardized  Root  Mean  Square 

Residual 

0.06 
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Table  31 

Mean  Navy  Climate  I  ndex  scores  by  demographic  characteristics 


Characteristic 

Mean  (SD) 

P 

Gender 

<  .0001 

Male 

61.8  (16.2) 

Female 

58.1  (16.8) 

Education 

<  .0001 

High  school  or  less 

59.1  (17.5) 

Some  college  or  2-year  degree 

60.4  (16.8) 

Bachelor's  degree  or  more 

63.3  (14.9) 

Marital  Status 

<  .0001 

Married 

63.3  (15.7) 

Not  married 

57.8  (16.8) 

Children  under  21  Living  in  Household 

<  .0001 

Yes 

62.9  (15.9) 

No 

59.3  (16.7) 

Pay  Grade 

<  .0001 

Enlisted 

59.9  (17.1) 

Warrant  Officer 

65.8  (13.2) 

Officer 

63.5  (14.5) 

First  Enlistment  or  Term  of  Service  in  Navy 

<  .0001 

Yes 

56.3  (17.1) 

No 

64.4  (15.0) 

Currently  on  deployment 

.0007 

Yes 

57.9  (18.1) 

No 

61.4  (16.2) 
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Validation  of  Navy  Climate  I  ndex 

The  NPS  includes  the  following  two  items  which  ask  survey  respondents  for  their 
ratings  of  tone  which  is  defined  in  the  survey  as  “Tone  is  an  overall  measure  of  how 
Sailors  feel  about  the  Navy.” 

•  Q29A:  How  would  you  rate  Navy  tone? 

•  Q29B:  How  would  you  rate  your  current  command’s  tone? 

Each  item  includes  five  response  categories  from  1  (very  high)  to  5  (very  low).  To 
assess  the  construct  validity  of  the  NCI,  NCI  scores  by  respondents’  ratings  on  these  two 
items  were  compared.  As  shown  in  Figures  101  andi02,  index  scores  increase  with  more 
positive  respondent  ratings  of  Navy  tone  and  their  current  command’s  tone  (p  <  .0001). 

In  addition,  it  was  hypothesized  that  NCI  scores  would  be  positively  related  to  the 
following  item  on  job  satisfaction 

•  Q65:  Considering  everything,  how  satisfied  are  you  with  Navy  life? 

As  expected,  scores  on  the  NCI  increased  with  greater  satisfaction  with  Navy  life  (p  < 
.0001;  see  Figure  103). 

Finally,  relationship  between  NCI  scores  and  three  items  measuring  retention 
intentions  was  evaluated: 

•  Q36A:  I  plan  to  serve  out  my  current  term  of  service  or  obligation. 

•  Q36B:  I  plan  to  reenlist  or  continue  with  my  career  in  the  Navy  at  my  next 

decision  point. 

•  Q36C:  I  plan  to  stay  in  the  Navy  for  a  full  career  if  possible. 

While  the  mean  NCI  scores  varied  significantly  across  all  three  retention  intention 
items  (p  <  .0001),  the  pattern  of  means  in  the  figures  suggests  that  the  relationship  is 
not  as  strong  for  item  Q36A  (intentions  to  serve  out  current  term  of  service  or 
obligation)  as  the  other  two  items  (see  Figures  104  to  106).  Perhaps  this  is  due  to  the 
reduced  amount  of  choice  and  alternatives  involved  in  serving  out  current  obligations  in 
comparison  to  making  choices  at  future  decision  points. 
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Q29A:  How  would  you  rate  Navy  tone? 


Figure  101.  Mean  Navy  Climate  I  ndex  scores  by  respondent  ratings  of  Navy 

tone. 
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Q29B:  How  would  you  rate  your  current  command's  tone? 


Figure  102.  Mean  Navy  Climate  I  ndex  scores  by  respondent  ratings  of 

command's  tone. 
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Q65:  Considering  everything,  how  satisfied  are  you  with  Navy  life? 


Figure  103.  Mean  Navy  Climate  I  ndex  scores  by  respondent  ratings  of 

satisfaction  with  Navy  life. 
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53.33 


51.95 


54.74 


55.63 


Strongly  Disagree  Neither  agree  Agree  Strongly 

disagree  nor  disagree  agree 

Q36A:  I  plan  to  serve  out  my  current  term  of  service  or  obligation 


Figure  104.  Mean  Navy  Climate  I  ndex  scores  by  retention  intentions: 
Plans  to  serve  out  current  term  of  service  or  obligation. 
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Figure  105.  Mean  Navy  Climate  I  ndex  scores  by  retention  intentions: 
Plans  to  reenlist  or  continue  career  with  Navy  at  next  decision  point. 
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Figure  106.  Mean  Navy  Climate  I  ndex  scores  by  retention  intentions: 
Plans  to  stay  in  Navy  for  a  full  career. 
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Re- Enlistment/  Continuation  I  ntention  I  ndex 


The  items,  scales,  and  NCI  described  in  previous  chapters  provide  comprehensive 
and  useful  information  concerning  Sailor  perceptions  of  Navy  life.  However,  in  some 
cases  it  may  not  be  feasible  to  administer  all  101  items  comprising  the  18  NPS  scales  or 
even  the  48  items  included  in  the  NCI.  Therefore,  a  smaller  subset  of  items  was  explored 
to  determine  if  those  items  may  provide  useful  information  concerning  Sailor 
perceptions  of  Navy  life  when  circumstances  do  not  permit  the  inclusion  of  the  larger  set 
of  items.  One  of  the  primary  outcomes  related  to  perceptions  of  Navy  life  is  a  Sailor’s 
intentions  to  re-enlist  (enlisted)  or  continue  (officers)  in  the  Navy  at  their  next  decision 
point.  In  this  chapter,  an  index  of  a  sub-set  of  NPS  items  was  developed  to  predict  re- 
enlistment/continuation  intentions 

Re-  Enlistment/  Continuation  I  ntention  I  ndex  for  All  Sailors 

A  cross-validation  approach  was  used  for  developing  and  validating  the  re¬ 
enlistment/continuation  approach.  Specifically,  the  sample  of  3,610  participants  was 
randomly  divided  into  two  halves  with  1,805  participants  each.  The  first  half  of  the 
sample  was  used  to  develop  the  index  and  second  half  of  the  sample  was  used  to  validate 
the  index.  In  the  remainder  of  this  chapter,  these  two  halves  of  the  sample  are  referred 
to  as  the  development  sample  and  the  validation  sample. 

Using  data  from  the  development  sample,  the  responses  to  item  Q36B  (I  plan  to  re¬ 
enlist  or  continue  my  career  with  the  Navy  at  my  next  decision  point)  were  collapsed 
into  2  categories:  intend  to  re-enlist/continue  (agree  or  strongly  agree)  and  does  not 
intend  to  re-enlist/continue  (disagree,  strongly  disagree,  or  neither  agree  nor  disagree). 
All  101  items  from  the  original  NPS  scales  were  entered  into  a  backward  stepwise 
logistic  regression  as  possible  predictors  of  re-enlistment/continuation  intentions. 

Items  on  overall  ratings  of  morale,  tone,  job  satisfaction,  and  satisfaction  with  Navy  life 
and  several  demographics  and  job-related  variables,  including  gender,  martial  status, 
education  level,  presence  of  children  under  age  21  in  household,  and  pay  grade  (enlisted, 
warrant  officer,  officer)  were  also  included. 

The  criterion  for  a  predictor  to  remain  in  the  model  was  a  p-value  of  0.10  or  less.  To 
avoid  the  elimination  of  cases  due  to  the  list  wise  deletion  method  used  for  missing 
values  in  the  backwards  stepwise  regression  procedure,  missing  values  were  recoded  to 
the  mid-point  for  each  scale  for  these  analyses.  In  other  words,  a  missing  value  for  items 
from  the  Morale  scale  would  be  recoded  to  the  category  of  “no  effect.” 

The  logistic  regression  results  for  this  model  are  shown  in  Table  32.  Based  on  the 
results  of  the  logistic  regression  analyses,  a  scoring  algorithm  for  the  re-enlistment/ 
continuation  intention  index  was  created,  which  involves  multiplying  the  items  by  their 
regression  coefficients  and  summing  them.  The  formula  for  calculating  the  re- 
enlistment/continuation  intention  index  is 
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Retention  =  (-o.3054*enlisted)+(o.2527*children)+(-o.6366*firstsrv)+ 

(-o.i542*Q8C)+(-o.i2ii*QioB)+(o.i9io*QioL)+(o.i7io*QioP)+ 
(-o.2994*Qi9B)+(-  o.1240*Q2iA)+(o.66o9*Q26A)+(o.3555*Q26D)+ 
(-o.2137*Q26F)+(o.2522*Q28C)+(o.1900*Q37A)+(-o.1372*Q37B)+ 
(-o.1438*Q37C)+(o.i831*Q37D)+(-o.1347*Q38A)+(o.3557*Q65); 

After  computing  the  index  scores,  receiver  operating  characteristic  (ROC)  curve 
analyses  was  then  conducted  (Hsiao,  Bartko  &  Potter,  1989;  Swets,  1995)  to  identify  a 
cut  point  for  the  scores  which  could  be  used  to  identify  which  Navy  personnel  are  likely 
to  re-enlist/continue.  ROC  analyses  provide  information  about  the  ability  of  the  index  to 
correctly  classify  personnel  into  those  who  intend  to  re-enlist/continue  and  those  who 
do  not.  The  proportion  of  people  who  intend  to  re-enlist/continue  who  are  correctly 
classified  as  intending  to  re-enlist/continue  based  on  the  index  (i.e.,  sensitivity)  and  the 
proportion  of  people  who  do  not  intend  to  re-enlist/continue  and  are  correctly  classified 
as  such  based  on  the  index  (i.e.,  specificity)  are  calculated  for  various  cut  points  of  the 
index.  These  values  are  then  used  to  compute  an  ROC  curve  which  plots  sensitivity 
versus  1  minus  specificity  (i.e.,  true  positive  proportion  versus  false  positive  proportion) 
for  the  possible  cut  points  of  the  test.  The  area  under  the  ROC  curve  measures  the 
overall  accuracy  of  the  index.  An  area  of  1.0  indicates  the  most  accurate  index  possible 
whereas  an  area  of  0.5  reflects  accuracy  no  greater  than  chance. 

Based  on  the  ROC  analyses,  a  cutoff  (i.e.,  cut)  score  for  each  index  was  selected  that 
maximized  sensitivity  after  achieving  a  specificity  level  of  at  least  0.7.  The  positive 
predictive  power  (PPP)  and  negative  predictive  power  (NPP)  was  then  computed  at  the 
selected  cut  score.  The  PPP  represents  the  percentage  of  Sailors  who  are  classified  as 
intending  to  re-enlist/continue  based  on  the  index  cut  score  who  do  in  fact  intend  to  re- 
enlist/continue.  Conversely,  the  NPP  indicates  the  percentage  of  Sailors  who  do  not 
meet  the  cut  score  who  do  not  intend  to  re-enlist/continue. 

Table  33  presents  the  areas  under  the  ROC  curve,  the  cut  scores,  and  corresponding 
statistics  for  the  re-enlistment/continuation  intention  index.  As  shown  in  the  table,  the 
index  performed  moderately  well  with  an  area  under  the  ROC  curve  (AUC)  of  0.82, 
sensitivity  of  0.78,  and  specificity  of  0.71. 

To  validate  the  re-enlistment/continuation  intention  index,  the  scoring  algorithms 
were  applied  and  cut  points  identified  using  the  development  sample  to  the  validation 
sample.  As  shown  in  Table  33,  slightly  lower  values  were  found  for  this  sample:  AUC  = 
0.79,  sensitivity  =  0.74,  and  specificity  =  0.68. 

Term- Specific  Re- Enlistment/ Continuation  Intention  Index 

In  the  regression  model  for  all  Sailors  described  above,  whether  or  not  a  Sailor  was 
in  his  or  her  first  term  of  service  was  a  highly  significant  predictor  of  re¬ 
enlistment/continuation  intentions  (p  <  .0001).  Given  these  results,  further  analyses 
were  conducted  to  explore  whether  predictors  of  re-enlistment/continuation  intention 
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may  vary  according  to  whether  a  Sailor  is  in  his  or  her  first  term  of  service.  Therefore, 
separate  models  for  those  in  their  first  term  or  service  versus  those  in  later  terms  of 
service  were  conducted  (Question  32). 

Tables  34  and  35  present  the  final  logistic  regression  model  for  Sailors  by  their  term 
of  service  based  on  the  developmental  sample.  Using  the  regression  coefficients  from 
the  two  models,  a  scoring  algorithm  for  the  re-enlistment/continuation  intention  index 
by  term  of  service  was  computed.  The  formula  for  calculating  the  re- 
enlistment/continuation  intention  index  for  Sailors  in  their  first  term  of  service  is: 

Retention  =  (-o.70*enlisted)+(-o.42*Qi9B)+(o.26*Qi9C)+(-o.53*Q2ia)+ 
(o.37*Q24c)+(o.65*Q24d)+(-o.73*Q24f)+(o.85*Q26a)+ 
(o.45*Q26d)+(-o.50*Q26f)+(o.33*Q37a)+(-o.48*Q37b)+ 
(o.39*Q37d)+(-o.2i*Q38a)+(o.50*Q65); 

Similarly,  the  re-enlistment/continuation  intention  index  scores  for  Sailors  in  a  later 
term  of  service  may  be  calculated  as  follows: 

Retention  =  (-o.23*Q8c)+(o.22*Qiol)+(o.23*Qiop)+(-o.i8*Qi3e)+ 
(-o.2i*Qi9b)+(o.i5*Q24e)+(-o.3i*Q25c)+(o.46*Q26a)+ 
(o.27*Q26d)+(o.34*Q28c)+(-o.22*Q38a)+(o.38*Q38d)+ 
(o.33#Q65); 

Table  36  presents  the  areas  under  the  ROC  curve,  the  cut  scores,  and  corresponding 
statistics  for  the  re-enlistment/continuation  intention  index.  As  shown  in  the  table,  the 
index  performed  very  well  at  predicting  re-enlistment/continuation  intentions  for 
Sailors  in  their  first  term  of  service  with  an  area  under  the  ROC  curve  (AUC)  of  0.87, 
sensitivity  of  0.86,  and  specificity  of  0.70.  The  index  was  slightly  less  able  to  predict  re- 
enlistment/continuation  intentions  for  those  in  later  terms  of  service  with  AUC  of  0.78, 
sensitivity  of  0.74,  and  specificity  of  0.70.  The  values  were  very  similar  for  the  validation 
sample,  supporting  the  generalizability  of  the  findings.  The  area  under  the  ROC  curve 
for  the  index  among  first-term  Sailors  in  the  validation  sample  was  0.84  and  among 
later-term  Sailors  was  0.74. 
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Table  32 

Logistic  regression  model  predicting  re-enlistment/  continuation  intention 
among  all  Sailors:  Development  sample 


Variable 

B 

SE 

OR 

95%  Cl 

P 

Enlisted 

-0.33 

0.13 

0.72 

0.56,  0.93 

.0108 

Have  children 

0.29 

0.13 

1.33 

1.03,  1.72 

.0268 

First  term  of  service 

-0.87 

0.15 

0.42 

0.31,  0.56 

<  .0001 

Q8C:  Adequate  spare  parts  and 
supplies 

-0.18 

0.06 

0.84 

0.75,  0.94 

.0027 

Q10B:  Performance  evaluation 
system 

-0.15 

0.07 

0.87 

0.76,  0.99 

.0316 

Q10L:  Pay,  bonuses,  other 
compensation 

0.21 

0.07 

1.23 

1.08,  1.40 

.0017 

Q10P:  Performance  of  the  crew  on 
exercises 

0.21 

0.08 

1.23 

1.05,  1.43 

.0085 

Q19B:  Time  spent  on  shore  duty 

-0.29 

0.06 

0.75 

0.66,  0.85 

<  .0001 

Q21A:  Career  gets  in  way  of 
personal  life 

-0.15 

0.06 

0.86 

0.77,  0.97 

.0127 

Q26A:  Feel  positive  about  future 
Navy  career 

0.55 

0.07 

1.74 

1.51,  1.99 

<  .0001 

Q26D:  Willing  to  change 
rating/designator  to  stay  in  Navy 

0.35 

0.05 

1.41 

1.29,  1.54 

<  .0001 

Q26F:  Concerned  future  policy 
changes  will  hurt  job 

-0.20 

0.06 

0.82 

0.73,  0.92 

.0005 

Q28C:  Would  recommend  the 

Navy  as  a  good  place  to  work 

0.26 

0.08 

1.29 

1.10,  1.52 

.0022 

Q37A:  Navy  has  personal  meaning 
for  me 

0.22 

0.10 

1.24 

1.03,  1.50 

.0229 

Q37B:  Feel  like  1  'm  part  of  the 
family  in  the  Navy 

-0.16 

0.09 

0.85 

0.71,  1.02 

.0744 

Q37C:  Feel  emotionally  attached  to 
the  Navy 

-0.16 

0.09 

0.86 

0.71,  1.02 

.0879 

Q37D:  Could  not  easily  become 
attached  to  another  organization 

0.22 

0.07 

1.24 

1.03,  1.50 

.0017 

Q38A:  Clear  understanding  of 
advancement/promotion  system 

-0.17 

0.07 

0.85 

0.74,  0.97 

.0143 

Q38D:  Expect  to  be  promoted 
within  current  term  of  service 

0.34 

0.05 

1.40 

1.27,  1.56 

<  .0001 

Q65:  Considering  everything,  how 
satisfied  are  you  with  Navy  life? 

0.34 

0.09 

1.40 

1.17,  1.69 

.0003 
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Table  33 

ROC  analysis  results  for  re-enlistment/  continuation 
intentions  among  all  Sailors 


Statistic 

Development 

Sample 

Validation 

Sample 

Area  under  ROC  curve 

0.82 

0.79 

Cut-off  score 

3.30 

3.30 

Sensitivity 

0.78 

0.74 

Specificity 

0.71 

0.68 

Positive  predictive  value 

0.73 

0.71 

Negative  predictive  value 

0.76 

0.71 
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Table  34 

Logistic  regression  model  predicting  re-enlistment/ continuation  intention 
among  Sailors  in  first  term  of  service:  development  sample 


Variable 

B 

SE 

OR 

95%  Cl 

P 

Enlisted 

Q19B:  Time  spent  on  shore 

-0.70 

0.27 

0.50 

0.29,  0.84 

.0091 

duty 

-0.42 

0.12 

0.66 

0.52.  0.83 

.0003 

Q19C:  Time  spent  on  sea  duty 
Q21A:  Career  gets  in  way  of 

0.26 

0.12 

1.30 

1.03,  1.65 

.0301 

personal  life 

-0.53 

0.11 

0.59 

0.48,  0.72 

<  .0001 

Q24C:  Deals  well  with  superiors 
Q24D:  Provides  adequate 

0.37 

0.18 

1.45 

1.01,  2.08 

.0434 

support  and  guidance 

Q24F:  Satisfied  with  command 

0.65 

0.20 

1.92 

1.31,  2.83 

.0009 

leadership 

Q26A:  Feel  positive  about 

-0.73 

0.20 

0.48 

0.33,  0.71 

.0002 

future  Navy  career 

Q26D:  Willing  to  change 
rating/designator  to  stay  in 

0.85 

0.12 

2.33 

1.83,  2.96 

<  .0001 

Navy 

Q26F:  Concerned  future  policy 

0.45 

0.08 

1.57 

1.34,  1.83 

<  .0001 

changes  will  hurt  job 

Q37A:  Navy  has  personal 

-0.50 

0.11 

0.61 

0.49,  0.76 

<  .0001 

meaning  for  me 

Q37B:  1  feel  like  1  'm  part  of  the 

0.33 

0.15 

1.39 

1.03,  1.87 

.0293 

family  in  the  Navy 

Q37D:  1  could  not  easily 
become  attached  to  another 

-0.48 

0.14 

0.62 

0.47,  0.82 

.0008 

organization 

Q38A:  Clear  understanding  of 
advancement/promotion 

0.39 

0.12 

1.48 

1.16,  1.89 

.0017 

system 

Q65:  Considering  everything, 
how  satisfied  are  you  with 

-0.21 

0.12 

0.81 

0.64,  1.03 

.0794 

Navy  life? 

0.50 

0.15 

1.65 

1.22,  2.23 

.0011 
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Table  35 

Logistic  regression  model  predicting  re-enlistment/ continuation  intention 
among  Sailors  in  later  term  of  service:  development  sample 


Variable 

B 

SE 

OR 

95%  Cl 

P 

Q8C:  Adequate  spare  parts 
and/or  supplies 

-0.23 

0.07 

0.79 

0.69,  0.91 

.0011 

Q10L:  Pay,  bonuses,  other 
compensation 

0.22 

0.08 

1.24 

1.05,  1.47 

.0097 

Q10P:  Performance  of  the 
crew,  work  team,  or  ship  on 
exercises 

0.23 

0.10 

1.26 

1.03,  1.53 

.0224 

Q13E:  Feeling  of 
accomplishment!  get  from 
doing  my  job 

-0.18 

0.08 

0.83 

0.71,  0.97 

.0205 

Q19B:  Time  spent  on  shore 
duty 

-0.21 

0.08 

0.81 

0.70,  0.95 

.0070 

Q24E:  Responsive  to  Sailor 
needs  and  concerns 

0.15 

0.09 

1.16 

0.97,  1.39 

.0971 

Q25C:  Command  leadership 
communicates  positive 
attitude  about  Navy 

-0.31 

0.11 

0.74 

0.59,  0.92 

.0062 

Q26A:  1  feel  positive  about  my 
future  Navy  career 

0.46 

0.09 

1.58 

1.33,  1.87 

<  .0001 

Q26D:  1  would  be  willing  to 
change  my  rating/designator 
if  it  was  the  only  way  1  could 
stay  in  the  Navy 

0.27 

0.06 

1.31 

1.17,  1.46 

<  .0001 

Q28C:  1  would  recommend  the 
Navy  as  a  good  place  to  work 

0.34 

0.10 

1.40 

1.15,  1.72 

.0010 

Q38A:  1  have  a  clear 
understanding  of  the  present 
Navy 

advancement/promotion 

system 

-0.22 

0.09 

0.80 

0.68,  0.95 

.0117 

Q38D:  1  expect  to  be 
advanced/promoted  within 
my  current  term  of  service, 
commitment,  or  obligated 
service 

0.38 

0.06 

1.47 

1.30,  1.65 

<  .0001 

Q65:  Considering  everything, 
how  satisfied  are  you  with 

Navy  life? 

0.33 

0.12 

1.39 

1.10,  1.75 

.0053 
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Table  36 

ROC  analysis  results  for  re-enlistment/  continuation  intentions  by  term  of 

service 


First  Term  of  Service 

Later  Term  of  Service 

Statistic 

Development 

Sample 

Validation 

Sample 

Development 

Sample 

Validation 

Sample 

Area  under  ROC  curve 

0.87 

0.84 

0.78 

0.74 

Cut-off  score 

2.54 

2.54 

3.95 

3.95 

Sensitivity 

0.86 

0.84 

0.74 

0.70 

Specificity 

0.70 

0.72 

0.70 

0.64 

Positive  predictive  value 

0.61 

0.65 

0.79 

0.75 

Negative  predictive  value 

0.91 

0.88 

0.63 

0.58 
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1  -  Specificity 

Diagonal  segments  are  produced  by  ties. 


Figure  107.  ROC  curve  for  Retention  I  ndex  among  all  Sailors: 

Development  sample. 


1  -  Specificity 

Diagonal  segments  are  produced  by  ties. 

Figure  108.  ROC  curve  for  Retention  I  ndex  among  all  Sailors: 

Validation  Sample. 
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1  -  Specificity 

Diagonal  segments  are  produced  by  ties. 


Figure  109.  ROC  curve  for  Retention  I  ndex  among  Sailors  in  first  term  of 

service:  Development  Sample. 


1  -  Specificity 

Diagonal  segments  are  produced  by  ties. 

Figure  110.  ROC  curve  for  Retention  I  ndex  among  Sailors  in  later  term  of 

service:  Development  Sample. 
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1  -  Specificity 

Diagonal  segments  are  produced  by  ties. 


Figure  111.  ROC  curve  for  Retention  I  ndex  among  Sailors  in  first  term  of 

service:  Validation  Sample. 


1  -  Specificity 

Diagonal  segments  are  produced  by  ties. 

Figure  112.  ROC  curve  for  Retention  I  ndex  among  Sailors  in  later  term  of 

service:  Validation  Sample. 
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Conclusion  and  Discussion 


This  report  describes  our  psychometric  evaluation  of  existing  items  and  scales  on  the 
2005  NPS.  Overall,  the  psychometric  analyses  suggest  that  most  of  the  scales  performed 
very  well,  demonstrating  good  internal  consistency  and  construct  validity.  The  vast 
majority  of  items  had  high  levels  of  discrimination  and  a  spread  of  threshold 
parameters. 

However,  the  results  suggested  potential  improvements  to  six  of  the  scales.  Based  on 
the  item  and  scale  analyses,  as  well  as  item  content,  it  is  recommend  that  items  be 
removed  from  the  Workplace  Climate,  Communication,  Job  Security, 
Advancement/Promotion,  and  Detailing  scales.  In  addition,  it  is  recommended  that  the 
Career  Development  scale  be  divided  into  two  scales  measuring  Career  Progression  and 
Counseling  or  Guidance.  The  analyses  suggest  that  these  changes  will  improve  the 
psychometric  properties  of  the  scales  on  future  administrations  of  the  survey. 

An  additional  goal  of  the  current  investigation  was  to  develop  an  index  measuring 
overall  Navy  Climate.  The  Navy  Climate  Index  (NCI)  was  developed  and  included  the 
following  seven  scales  from  the  NPS:  (1)  workplace  climate,  (2)  organizational 
commitment,  (3)  morale,  (4)  job  security,  (5)  communication,  (6)  fairness,  and  (7)  Navy 
image.  A  second-order  confirmatory  factor  analysis  verified  the  factorial  validity  of  the 
NCI.  Scores  on  the  NCI  were  strongly  related  to  respondents’  ratings  of  Navy  tone,  their 
current  command’s  tone,  and  overall  satisfaction  with  Navy  life,  supporting  the 
construct  validity  of  the  index.  Demographic  comparisons  suggested  that  the  following 
factors  were  associated  with  more  positive  perceptions  of  Navy  Climate:  male,  higher 
education,  married,  children  under  age  21  in  household,  officer  or  warrant  officer,  not  in 
first  term  of  service,  and  not  currently  deployed.  The  NCI  was  significantly  related  to 
intentions  to  re-enlist  (or  continue)  and  to  stay  in  the  Navy  for  a  full  career. 

Next,  an  index  for  predicting  intentions  to  re-enlist  or  continue  at  the  next  decision 
point  was  developed.  A  cross-validation  approach  was  utilized  wherein  the  NPS  sample 
was  randomly  split  in  half  and  the  index  was  developed  on  one  half  and  validated  on  the 
other.  The  index  was  developed  using  backwards  stepwise  logistic  regression  to  identify 
items  and  demographic  and  job  characteristics  which  predicted  reenlistment  or 
continuation  intentions.  The  results  suggested  that  models  predicting  reenlistment  or 
continuation  intentions  separately  for  Sailors  in  their  first  term  of  service  and  those  in 
later  terms  of  service  were  more  accurate  than  a  model  developed  for  all  Sailors 
combined.  Cut  points  for  the  reenlistment  or  continuation  intention  index  were 
identified  using  ROC  analysis.  The  index  had  an  overall  sensitivity  of  0.85  and 
specificity  of  0.71  for  Sailors  in  their  first  term  of  service  when  the  full  sample  was  used. 
Values  for  Sailors  in  later  terms  of  service  were  somewhat  lower  with  sensitivity  =  0.72 
and  specificity  =  0.67. 

A  limitation  of  our  study  was  that  only  data  on  retention  intentions  rather  than 
actual  retention  behavior  was  available.  Very  few  respondents  are  likely  to  have  made 
retention  decisions  during  the  short  time  frame  of  the  study,  preventing  us  from  using 
retention  behaviors  as  a  criterion  for  developing  the  index.  The  reenlistment  or 
continuation  intention  index  developed  in  this  study  will  need  to  be  tested  as  a  predictor 
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of  the  actual  behavior  of  Sailors  when  given  the  choice  of  re-enlisting  or  continuing  their 
careers.  Work  currently  being  conducted  at  NPRST  with  prior  year  NPS  data  suggests 
that  reenlistment  or  continuation  intentions  are  solid  predictors  of  subsequent 
reenlistment  or  continuation  behaviors.  This  administration  of  the  NPS  will  be 
combined  with  others  surveys  that  ask  about  career  intentions  so  that  they  can  be 
followed-up  across  time  to  assess  the  strength  of  the  relationship  between  career 
intentions  and  actual  behavior. 

Future  Directions 

The  scales  that  were  developed  and  validated  in  this  study  (Navy  Climate  Index  and 
Reenlistment  or  Continuation  Intention  Index)  provide  useful  tools  for  assessing 
Sailors’  perceptions  of  their  Navy  careers.  Future  research  should  further  enhance  the 
usability  of  these  scales.  In  particular,  the  number  of  items  on  the  scales  and  indices 
could  potentially  be  reduced  further  by  eliminating  similar  or  redundant  items.  For 
example,  the  Rasch  partial-credit  model,  a  type  of  IRT  model,  could  be  used  to  identify 
items  that  have  similar  threshold  parameters.  If  a  scale  contains  two  items  with  the 
same  threshold  parameters,  one  item  could  be  removed  with  minimal  or  no  loss  of 
information. 

An  alternative  approach  to  scale  reduction  is  to  explore  the  suitability  of  individual 
items  as  proxies  for  scales.  For  example,  the  NPS  includes  an  item  that  asks  respondents 
to  rate  the  overall  morale  of  their  present  command.  It  is  possible  that  this  item  could  be 
used  as  a  proxy  for  the  entire  16-item  morale  scale.  Regression  analyses  could  be 
conducted  to  determine  the  amount  of  variance  in  scores  on  the  morale  scale  that  is 
accounted  for  by  the  global  morale  item,  allowing  us  to  quantify  the  potential  loss  of 
information  by  using  a  single-item. 

It  is  important  to  note  that  reducing  the  number  of  items  on  scales  may  also  reduce 
the  reliability  or  validity  of  the  scales.  However,  in  some  circumstances,  shorter  scales 
may  provide  benefits,  such  as  lower  respondent  burden  and/or  reduced  time  and  costs 
for  survey  administration,  which  may  outweigh  some  loss  in  reliability  or  validity.  For 
example,  a  short  NCI  could  be  administered  as  part  of  a  Navy  Quick  Poll  to  provide 
more  frequent  statistics  on  current  Navy  climate,  allowing  leaders  to  rapidly  identify 
concerns  and  make  changes.  A  shortened  index  could  also  be  adapted  for  future  use 
with  the  Navy  Reserve  Component  as  well  as  the  Navy  civilian  workforce  allowing  a 
Total  Force  Climate  Index  that  would  assess  satisfaction  and  related  work  and  Navy  life 
outcomes. 

Future  work  should  utilize  psychometric  methods  to  establish  the  significance  of 
changes  in  index  scores  in  terms  of  important,  practical  outcomes,  such  as  reenlistment 
or  continuation  intentions.  These  methods  would  associate  a  specific  increase  in  the 
value  of  the  index  to  a  change  in  a  particular  outcome.  For  example,  methods  have  been 
utilized  in  health-related  quality  of  life  research  to  establish  what  is  referred  to  as  a 
minimally  clinically  important  difference  (MCID)  for  a  quality  of  life  scale.  An  MCID  has 
been  defined  as  the  “smallest  difference  in  scores  in  the  domain  of  interest  which 
patients  perceive  as  beneficial  and  which  would  mandate,  in  the  absence  of  troublesome 
side  effects  and  excessive  cost,  a  change  in  the  patient’s  management”  (Jaeschke,  Singer, 
&  Guyatt,  1989). 


126 


While  not  directly  transferable,  the  MCID  terminology  is  conceptually  similar  to  the 
NCI  and  the  statistical  techniques  can  be  used  to  identify  what  change  in  NCI  scores  is 
associated  with  a  change  in  retention  intentions  (i.e.,  changing  from  intending  to  leave 
the  Navy  to  intending  to  continue).  For  example,  an  anchor-based  approach  to 
determining  MCID  may  be  used  to  determine  the  average  change  in  NCI  scores 
corresponding  with  an  increase  from  “agree”  to  “strongly  agree”  on  item  Q36B  (I  plan  to 
reenlist  or  continue  with  my  career  in  the  Navy  at  my  next  decision  point).  Future 
studies  should  also  explore  the  relationship  between  NCI  scores  and  outcomes  from 
data  sources  external  to  the  NPS,  such  as  performance  measures, 
advancements/promotions,  or  test  (Armed  Services  Vocational  Aptitude  Battery  or 
ASVAB)  and  training  scores. 
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